A HISTORY OF STUDENT RATINGS: Meso-Responserate Era (2000–2010)
The first decade of the new millennium rode on the search engines of the previous decade. A bunch of publications kicked it off with half a dozen edited volumes on student ratings and faculty evaluation published by Jossey-Bass in their Teaching and Learning series (Ryan, 2000, #83; Lewis, 2001, #87; Knapper & Cranton, 2002, #88; Sorenson & Johnson, 2004, #96) and Institutional Research series (Theall, Abrami,& Mets, 2001, #109; Colbeck, 2002, #114).
There were no planet-shattering technical developments, although there were several software options created specifically for online administration and reporting. Only a trickle (OOPS! Sorry. This is just residual fluid from the previous metaphor.) of articles on midterm formative ratings and other topics appeared. Finally, three books hit the Amazon pages in the last half of the decade: Arreola’s (2007)14th edition of his popular work, Seldin’s (2006) 128th edited volume, and our hero’s (Moi, 2006) psychometric-humorous attempt (see References in next blog).
Most of the activity and discourse on student ratings concentrated on practical issues. There were several trends that continued from the previous decade:
(1) student ratings data were being supplemented with other data, particularly peer review of teaching and course materials and letters of recommendation by each professor’s mommy or daddy, for decisions about teaching effectiveness;
(2) institutions reviewed the quality of their tools to consider either a commercial package, such as THOUGHT, or develop their own “homegrown” scales with online reporting by Academic Management Systems or other support;
(3) the technical quality of many “homegrown” scales prompted me to coin the new psychometric term "putrid"; and
(4) the debate over paper-based vs. online administration grew with cost and student response rates as the deal-breakers. Online packages were available everywhere. The importance of response rates to the validity of student ratings became so critical to the adoption of an online system that this era was named the "Meso-Responserate Era."
We’ve come to the end of this series. I’ve run out of jokes. The last blog will be an epilogue of a few final thoughts and jokes.
COPYRIGHT © 2010 Ronald A. Berk, LLC
State-of-the-Art of Student Ratings
There is more research on student ratings than any other topic in higher education. More than 2,500 publications and presentations have been cited over the past 90 years. Those ratings have dominated as the primary and, frequently, only measure of teaching effectiveness at colleges and universities for the past five decades. In fact, the evaluation of teaching has been in a metaphorical cul-de-sac with student ratings as the universal barometer of teaching performance. And, if you’ve ever been in a cul-de-sac or metaphor, you know what that’s like. OMGosh, it can be Stephen Kingish terrifying.
In surveys over the past decade, it was found that 86% of U.S. liberal arts college deans and 97% of department chairs use student ratings for summative decisions about faculty. Only recently has there been a trend toward augmenting those ratings with other sources of evidence and better metaphors (Arreola, 2007; Berk, 2006; Knapper & Cranton, 2001; Seldin, 2006).
So how in the ivory tower did we get to this point? Let’s trace the major historical events. Hold on to your online administration response rates. Here we go.
A History of Student Ratings
This history covers a timeline of approximately 100 billion years, give or take a day or two, ranging from the age of dinosaurs to the age of Conan O’Brien’s new cable TV show. Obviously, it’s impossible to squish every event that occurred during that period in this series. Instead, that span is partitioned into six major eras within which salient student-ratings activities are highlighted. A blog will be devoted to each of those eras.
References
Arreola, R. A. (2007). Developing a comprehensive faculty evaluation system (3nd ed.). San Francisco: Jossey-Bass.
Berk, R. A. (2006). Thirteen strategies to measure college teaching. Sterling,VA: Stylus.
Knapper, C., & Cranton, P. (Eds). (2001). Fresh approaches to the evaluation of teaching (New Directions for Teaching and Learning, No. 88). San Francisco: Jossey-Bass.
Seldin, P. (Ed.). (2006). Evaluating faculty performance. San Francisco: Jossey-Bass.
My 1st era blog will tackle prehistoric student ratings of the “Meso-Pummel Era.” How did cave men and women measure teaching performance? Their methods were a bit crude, but effective.
COPYRIGHT © 2010 Ronald A. Berk, LLC
