Interviews
MIST conducts interviews twice a year. In the early fall, we interview key district leaders (e.g., Chief Academic Officer, head of Curriculum and Instruction, head of Mathematics Department, head of Office of Leadership, head of Office of English Language Learners) to find out the district’s theory of action for improving middle-grades mathematics instruction for the academic year (i.e., the intended design).
In January, we interview teachers, coaches, principals, and district leaders to find out how the theory of action is playing out (i.e., the enacted design). Interviews are customized for each role group for each district, depending on the district’s theory of action. We have included generic interviews below for Teacher, Coach, and Principal as well as for several district offices (Office of Curriculum and Instruction, Office of Leadership, Mathematics Department, Office of English Language Learners, Special Education, Office of Research, Evaluation, and Accountability). Questions about networks (teacher, coach, principal) were adapted from interview protocols provided by Cynthia Coburn and Jennifer Russell.
Theory of Action (Fall)
This protocol includes questions for finding out about key aspects of a district’s design for improving middle-grades mathematics teaching and learning. This protocol is customized based on each district’s plans and is revised each year.
Teacher Interview (January)
This protocol focuses on the teacher’s teaching responsibilities, opportunities for teacher collaboration, teacher’s views on high-quality instruction in mathematics, how the teacher works with the coach and principal (or assistant principal), and the supports and resources that have been provided to the teacher. This protocol is then customized based on each district’s theory of action and is revised each year.
Coach Interview (January)
This protocol focuses on the coach’s role, how the coach works with teachers and administrators, the coach’s views on high-quality instruction in mathematics, and the supports and resources that have been provided to the coach. This protocol is then customized based on each district’s theory of action and is revised each year.
Principal Interview (January)
This protocol focuses on the principal’s understanding of the district’s theory of action for instructional improvement in middle-grades mathematics, his/her goals and the current plan for improvement in mathematics teaching in the school, to whom the principal is accountable and for what, principal’s views on high-quality instruction in mathematics, how the principal works with the coach, and the supports and resources that have been provided to the principal. This protocol is then customized based on each district’s theory of action and is revised each year.
District Leaders Interviews (January)
We interview several key district leaders each January to find out how the improvement efforts in middle-grades mathematics are progressing. These protocols focus on the leader’s role in the district, the current middle school math initiative(s), the leader’s views on high-quality instruction and high-quality instructional leadership, and how the different units in the district office work together to promote instructional leadership and improvement in middle school mathematics instruction. We created protocols for leaders in each office that had a stake in the middle-grades mathematics improvement efforts (e.g., Office of Curriculum and Instruction, Office of Leadership, Mathematics Department, Office of English Language Learners, Special Education, Office of Research, Evaluation, and Accountability). Sample protocols are included below. Each of these protocols is then customized based on each district’s theory of action and is revised each year.
Surveys
The teacher, principal, and coach surveys serve as one part of the project’s quantitative data collection. The overall goal of the quantitative data analysis is to test hypotheses about associations between institutional support structures and changes in teachers’ content knowledge for teaching, their instructional practices, and student achievement. The surveys provide repeated measures of support structures operating at the school-level.
The teacher survey measures teachers’ perceptions of the support structures for developing ambitious instructional practices in mathematics. It includes items concerning the level of teachers’ participation in learning communities and informal networks, their propensity to seek instructional advice, and who they seek advice from (e.g., math coaches); the degree to which the interactions between teachers in these learning communities and networks focus on central mathematical ideas and how to relate them to students’ reasoning; the degree to which teachers and formal and informal leaders have a shared vision for math instruction and student learning; the degree to which instructional leadership is distributed among formal and informal leaders; and the professional development that teachers have received in support of improved instructional practices in mathematics.
The principal survey includes items concerning the assistance provided by mathematics coaches to support the principal’s work as an instructional leader in mathematics; the principal’s work to both support teachers and hold them accountable for developing ambitious instructional practices in mathematics; and the professional development that the principal has received regarding the district’s instructional program in mathematics
The coach survey includes items concerning the coach’s work to both support teachers and hold them accountable for developing ambitious instructional practices in mathematics; assistance provided by mathematics coaches to support the principal’s work as an instructional leader in mathematics; and professional development that the coach has received regarding the district’s instructional program in mathematics.
The surveys include items developed and refined specifically for this project, and those already developed and field-tested in other research. Pre-existing items come from work by Bryk, Camburn, and Louis (1999), Bryk and Schneider (2002), Spillane (1996), the Consortium on Chicago School Research, the National Evaluation of the Eisenhower Professional Development Program, and the National Longitudinal Study of No Child Left Behind, Study of Instructional Improvement, and the Study of School Leadership. Some survey items about teacher learning communities and instructional leadership in mathematics come from an instrument developed by Spillane and colleagues (Distributed Leadership for Middle School Mathematics Education: Content Area Leadership Expertise in Practice, HER 0412510). A number of items developed for this project are based on the construct of the “Leadership Content Knowledge” (LCK) introduced by Stein & D’Amico (2000, April) and elaborated by Nelson (2005) and Stein and Nelson (2003). Items that had not been previously used were field-tested by conducting cognitive interviews of middle-school math teachers in two urban districts (American Statistical Association, 1997). This methodology is useful for identifying overly complex items, social desirability response bias, and unknowingly misleading responses (Biemer, Groves, Lyberg, Mathiowetz, & Sudman, 1991; Desimone & Le Floch, 2004).
Learning Mathematics for Teaching Web site
Video-Recordings of Classroom Instruction: Measures of Students' Opportunities to Learn Mathematics
MIST uses the Instructional Quality Assessment (IQA: Boston, 2012) instrument to measure the quality of students' opportunities to learn mathematics (as captured in video-recordings of classroom instruction). The Instructional Quality Assessment (IQA) is based on the Mathematical Tasks Framework and is consistent with the districts’ instructional visions and professional development programs. The IQA is designed to measure the cognitive demand of task as it appears in curricular materials, the cognitive demand of task as implemented, and the quality of concluding whole-class discussion. View IQA here.
Members of the MIST team, in consultation with Melissa Boston, developed additional rubrics to measure the quality of the setup phase of instruction (i.e., before the task is implemented) (Jackson, Garrison, Wilson, Gibbons, & Shahan, in press). View Set Up Rubrics here.
For more information on the IQA instrument, please contact:
Melissa Boston
Duquesne University
Asst. Professor of Mathematics Education
Dept. of Instruction and Leadership
bostonm@duq.edu
