Development of the Vanderbilt Assessment of Leadership in Education (VAL-ED)

Abstract

Meeting the challenge of raising student achievement and closing the achievement gaps in America's schools depends on school leaders who effectively guide instructional improvement.

Many districts use leadership assessment for formative or summative purposes. Whatever the purpose, there is general agreement that the current state of leadership assessment is lacking. The identification and development of effective school leaders has been significantly hampered by the lack of technically sound tools for assessing and monitoring leadership performance. The Vanderbilt Assessment of Leadership in Education (VAL-ED) fills this gap.

Theory

The VAL-ED is a paper and on-line assessment which utilizes a multi-rater, evidence-based approach to measure the effectiveness of school leadership behaviors known to influence teacher performance and student learning. The VAL-ED measures core components and key processes. Core components refer to characteristics of schools that support the learning of students and enhance the ability of teachers to teach. Key processes refer to how leaders create those core components.

High Standards for Student Learning—There are individual, team, and school goals for rigorous student academic and social learning.
Rigorous Curriculum (content)—There is ambitious academic content provided to all students in core academic subjects.
Quality Instruction (pedagogy)—There are effective instructional practices that maximize student academic and social learning.
Culture of Learning & Professional Behavior—There are integrated communities of professional practice in the service of student academic and social learning. There is a healthy school environment in which student learning is the central focus.
Connections to External Communities—There are linkages to family and/or other people and institutions in the community that advance academic and social learning.
Performance Accountability—Leadership holds itself and others responsible for realizing high standards of performance for student academic and social learning. There is individual and collective responsibility among the professional staff and students.

Key Processes Refer to How Leaders Create Those Core Components

Planning—Articulate shared direction and coherent policies, practices, and procedures for realizing high standards of student performance.
Implementing—Engage people, ideas, and resources to put into practice the activities necessary to realize high standards for student performance.
Supporting—Create enabling conditions; secure and use the financial, political, technological, and human resources necessary to promote academic and social learning.
Advocating—Promotes the diverse needs of students within and beyond the school.
Communicating—Develop, utilize, and maintain systems of exchange among members of the school and with its external communities.
Monitoring—Systematically collect and analyze data to make judgments that guide decisions and actions for continuous improvement.

Effective learning-centered leadership is at the intersection of the two dimensions: core components created through key processes. The conceptual framework for VAL-ED is based on a review of the learning-centered leadership research literature and alignment to the Interstate School Leaders Licensure Consortium (ISLLC) standards (see figure below). Every item in the Principal, Supervisor, and Teacher Response forms represents a cross-section of one core component and one key process.

Ease of Use

The VAL-ED instrument is easy for your school to adopt. Completing the VAL-ED, a 72-item inventory of behaviors, requires just 20-25 minutes per respondent. This can be done very easily at a faculty meeting. Or, the assessments can be handed out to teachers and completed on their own time; all that is needed is a faculty representative willing to pick up the assessments, put them in an envelope, and send them off! Because the teacher surveys are completely anonymous (no names required), there is no need to worry about loss of confidentiality. Additionally, schools or individual teachers can opt to take the VAL-ED online. This service will remind respondents with customized emails and allows them to complete the assessment on their own time in the privacy of their home or classroom. Respondents will be able to save their progress and log out at any time. Sample Set of Items Download PDF

360 Assessment

The VAL-ED is a 360° assessment. It is intended to be taken by the principal, the principal’s supervisor, and all teachers in the school. 360° assessments provide the best feedback to principals because they incorporate the input of all members of the school’s professional community. All respondents respond to the same items about principal leadership behaviors. When the principal receives a report with the results of the assessment, the report allows the principal to compare his or her own ratings on each of the core components/key processes against the ratings given by teachers and supervisors. In this way, the principal can get informative feedback about the leadership behaviors in which he or she is excelling and the behaviors on which more work is needed.
Principal Report 2008 (Sample 1) Download PDF

About the Assessment

The VAL-ED instrument consists of 72 items that comprise 6 core component subscales and 6 process subscales. There are two parallel forms (A and C) of the assessment to facilitate measuring growth over time. In this 360 degree evidenced-based assessment of leadership behaviors, each respondent rates the principal's effectiveness on a six point scale after having first indicated the sources of evidence on which the effectiveness is rated. The principal does not need to have performed the leadership behavior directly, but may have ensured that the behavior was done by others. The reference period is the current school year. See the sample instrument, which provides sample items and directions to the assessment.

The VAL-ED can be used as part of a comprehensive assessment of the effectiveness of a leader's behaviors by providing a detailed "picture" of perceived performance. When used appropriately, it yields valuable norm-referenced and criterion-reference scores for evaluating learning-centered leadership. It can be used annually or more frequently to measure performance growth, guide professional development, and facilitate a data-based performance evaluation.

All items and response scales were developed to be aligned with the ISLLC standards. Framework and assessment were critiqued by panels of education leaders and researchers. To establish content validity, items were independently sorted by three respondents into our 36-cell framework. Responses indicated strong content validity and guided further item revisions. Download PDF

Psychometrics

The VAL-Ed is designed, developed and tested to be both reliable (i.e. provide accurate measurement) and valid (i.e. measure leadership behaviors that lead to improved student achievement). To accomplish these goals, we followed a multi-stage development process that involved cognitive labs, pilot tests, and field tests. At each stage of the design and develop process, the properties of the instrument were investigated through empirical study and expert review. The process was guided by the American Educational Research Association's 1999 Standards for Educational and Psychological Testing. Responses indicated strong content validity and guided further item revisions.

Pilot/Cognitive Interviews: To investigate assessment format, item clarity, and item importance, two rounds of cognitive interviews were performed with panels of practicing educators. Responses indicated practitioner preferences and understanding about assessment format and item-level language.
Nine-School Pilot Test: To examine how VAL-ED works in a field setting, an urban district was recruited to participate in a pilot. Data from the pilot enabled us to begin establishing reliability for the twelve scales and construct validity for the assessment using factor analysis.
Cognitive Interviews of Online Prototype: To examine the utility of an online prototype of the VAL-ED and test changes made after the nine-school pilot, cognitive interviews were conducted with teachers, principals, and supervisor from three schools. Responses indicated a preference for the online assessment and helped identify important issues to address in future development.
Bias Review: The instrument was submitted to a bias review committee of urban district leaders to ensure the assessment does not advantage respondents for anything other than the leadership behaviors we seek to measure.
Field Test: In Spring, 2008, the instrument was field tested in approximately 300 schools. Responses were used to establish validity and scale reliability.
Proficiency Standards: A panel of education leaders was convened, using a holistic/body-of-work approach to set "national" proficiency standards based on pilot study/field test data. For more about the psychometric properties of the VAL-ED assessment, please see this document.

PROJECT WEBSITE

FUNDING AGENCY

Wallace Foundation

TOTAL DOLLARS

$1,500,000

PROJECT PERIOD

7/1/2005-6/30/2008

INVESTIGATORS

PI: Andrew Porter
Joseph Murphy
Ellen Goldring
Stephen N. Elliott

Purchase or request review copy of VAL-ED from Discovery Assessment.

Read about the new VAL-ED project