One critical psychometric issue neglected in the literature on administration of student rating scales and in wide-spread administration practices is the permissible window for completing the scales. This involves standardization of administration procedures. These procedures are required in the administration of all instruments, especially those used for personnel decisions in the Standards for Educational and Psychological Testing. Employment decisions about merit pay, pay cuts, promotion, demotion, and tenure must be based on evidence of teaching performance that meets the Standards and EEOC Uniform Guidelines on Employee Selection Procedures.
The validity of the responses, their comparability, and aggregate meaning based on group data hinge on WHEN the students complete the evals. The window must be NARROW, such as 48 hours immediately following the final exam or project submission. If there are items on the scale measuring student assessment procedures, administration practices, fairness, etc., the final exam must be completed before the scales can be administered. This STANDARDIZATION of scale administration is essential, whether online or paper-and-pencil, to insure the scores from all students have the same meaning. If a wide window is given where some students can complete rating scales before the final assessment and others after that assessment at their discretion, the validity of responses is shot to smithereens!
The issue is the link between the behaviors measured on the rating scale and the students' opportunity to render an accurate rating of each behavior? This is a validity concern. It is assumed that every student has had the same 45 hours (3-credit course) to observe those behaviors during the semester. If they miss a few classes, their evaluations should not be significantly affected.
COPYRIGHT © 2009 Ronald A. Berk, LLC
No comments:
Post a Comment