Philippine E-Journals

Alternatives to On-site Evaluations of the Performance of Student-Teachers: Issues in the Measurement of Inter-rater Agreement

Sivanes Phillipson | Shane N. Phillipson | Kim Fong Poon-mcbrayer

Discipline: Education

 

Abstract:

The evaluation of student-teacher performance during their field experience remains an important component of student-teachers’ professional training. However, increasing pressure on university faculty has prompted a search for alternatives to the practice of on-site visits for this evaluation. This research focused on the measurement issues related to the use of video recordings to evaluate student-teacher performance in a teacher education programme in Hong Kong. Sixteen student-teachers and three university faculty participated in the research, using a 17-item Field Experience Supervision Form as the basis for the evaluation. The first part of the study estimated intra-rater agreement of student-teacher performance in a live context compared with an evaluation based on a video recording. The second part estimated inter-rater agreement of student-teacher performance based solely on video recordings of lessons. Analysis of the responses using Gwet’s (2001) first-order agreement coefficient to estimate both intra- and inter-rater agreement of student-teacher performance, and a multi-facet Rasch model to study the interactions between student-teachers, raters, and medium, showed that video-recorded lessons can be an effective alternative to on-site evaluations.