Assessing educational leaders: New measurement tool aspires to make the process VAL-ED

topics: Education Reform

by Melanie Moran

For years, educational accountability has been associated with student achievement—and with the teachers who prepare those students. But as the No Child Left Behind act has singled out more and more schools for corrective action, attention has begun to move from teacher effectiveness in the classroom to the work being done in the principal’s office. A new assessment for principals developed by researchers at Vanderbilt University and the University of Pennsylvania has the potential to clarify what is meant by educational leadership and how it can be measured.

Discovery Education and Vanderbilt University are partnering to launch the Vanderbilt Assessment of Leadership in Education (VAL-ED), which is being exclusively distributed by Discovery Education.

“VAL-ED builds a picture of principal effectiveness by providing a detailed assessment of a principal’s perceived performance,” Camilla Benbow, Patricia and Rodes Hart Dean of Education and Human Development, said. "This assessment empowers administrators to effectively evaluate staff, diagnose strengths and weaknesses, and recommend pertinent professional development.”

VAL-ED was developed by Peabody faculty Joseph Murphy, Ellen Goldring and Stephen Elliott, and by Andrew Porter, dean of the University of Pennsylvania’s Graduate School of Education, to provide detailed, evidence-based assessment of principals’ performance. It is aligned with the national leadership standards set by the Interstate School Leaders Licensure Consortium, which Murphy helped to formulate.

The design, testing and dissemination of the instrument have been supported with a $1.5 million grant from the Wallace Foundation of New York. VAL-ED has been field tested in 100 elementary schools, 100 middle schools and 100 high schools in 53 districts and 27 states.

The assessment measures principal performance in six core components related to student achievement, for example, setting high standards for student learning, creating a culture of learning and professionalism, and offering quality instruction.

There are already numerous measures available for evaluating principals’ performance, and VAL-ED’s authors reviewed 79 of these in the early stages of the assessment’s development.

One result is that the researchers have taken pains to incorporate psychometric properties in the measurement tool, and they have worked with principals to obtain their feedback. As part of the development process, they conducted cognitive interviews, where principals were asked to talk out loud about their thought processes as they completed the evaluation. In a nine-school pilot test conducted in 2007, the instrument was found to be highly reliable.

“VAL-ED is a clear improvement over previous evaluation instruments,” Discovery Education Assessment Director William Dycus explained. “Many school districts have created various instruments to measure the performance of principals, but few have been as meticulously researched and rigorously tested. By providing accurate and reliable data, VAL-ED aids administrators in making decisions that impact student achievement.”

The measurement’s developers acknowledge that VAL-ED still has its limitations. For example, it only measures perceptions of a principal’s performance. They urge that supervisors also consider factors such as actual student-learning gains or graduation rates in their evaluations of principals.

“VAL-ED is notable in that it gives 360-degree feedback. It can be used annually to facilitate a data-based performance evaluation, or it can be used more frequently to measure performance growth or provide principals ongoing feedback throughout the school year,” said Joseph Murphy.

The instrument takes about 20-25 minutes to complete in either online or pencil and paper formats. To provide the 360-degree perspective on performance, teachers, the principal and the principal’s supervisor complete the evaluation. The assessment instrument is composed of 72 questions, in which each respondent is asked to evaluate the principal’s performance on 72 behaviors. Respondents rate performance on a scale from 1 for “ineffective” to 5 for “outstandingly effective” after considering data on which the evaluation is based, such as school documents or personal evaluation.

The instrument then reports results in two ways. First, it shows how the principal compares with a nationwide peer-group. Second, it portrays a principal’s performance as basic, proficient or distinguished, as determined by experts. The developers envision that districts using the measure will determine how much weight to give the instrument, or the scores of certain types of raters, as well as whether to emphasize performance against peers or performance against a standard.

How VAL-ED works

VAL-ED measures principals' performance at the intersection of six core components and six key processes. Each core component refers to school characteristics that support teaching and student learning.

> High standards for student learning: Goals are established for individuals, teams and the school for academic learning.

> Rigorous curriculum: The academic content in all core subjects, and provided to all students, is ambitious.

> Quality instruction: Teachers use instructional practices that are effective and maximize student learning.

> Culture of learning and professional behavior: Communities of professional practice are integrated in the school and serve student learning.

> Connections to external communities: Parents, families and external organizations are linked to the school and support academic and social learning.

> Performance accountability: Leadership holds itself and others responsible for student performance and meeting high standards. Professional staff and students see themselves as individually and collectively accountable.

Key processes refer to the specifics of how leaders create core components. The six key processes are planning, implementing, supporting, advocating, communicating and monitoring.

Respondents are asked to give effectiveness ratings to questions measuring 72 leadership behaviors. For each question, they are also asked to indicate the sources of evidence used in determining their rating.

Results are reported as summary profiles of the six core component scores compared to normative profiles of principals across the country as well as proficiency standards (basic, proficient, or distinguished).