“A FRACTURED, SEMI-FACTUAL HISTORY OF STUDENT RATINGS OF TEACHING: Meso-Golden Era (1970s)!”

A HISTORY OF STUDENT RATINGS: Meso-Golden Era (1970s)
The 1970s were called the “Golden Age of Student Ratings Research,” named after a bunch of “senior” professors who were doing research while living in assisted learning communities. Obviously, this period had to be named the "Meso-Golden Era." The research on the relationships between student ratings and instructor, student, and course characteristics burgeoned during these years. As the evidence on the validity and reliability of scales mounted, the use of the scales increased across the land, beyond East Lansing, in Nebraska (home of the Maryland Tar Heels).

To punctuate this burgeoning, commercially-produced, nationally-normed, student rating scales emerged during this era. Their superior technical quality and custom report forms compared to many “homegrown” tools provided colleges with the option of shopping for their rating scales in Ratings Mall. The first two scales were SIR (Educational Testing Service, 1972) and IDEA (Kansas State University, 1975). IAS Online (University of Washington) and SEEQ (Centre for Educational Assessment, Perth, Western Australia) became available later.

Student rating item banks also popped up during the ‘70s which permitted faculty to hand pick statements from a catalog to build their own customized scales. These banks were called POP-UPS PICES (Purdue University, 1974) and CIEQ (University of Arizona, 1977). Entering this commercial rating scale derby was not without challenges. Most ventures without barely pronounceable 3–5 letter acronym names tanked immediately.

These burgeoning golden years also witnessed the first 25 books on the topic. The winning prize for first book went to Richard Miller for Evaluating Faculty Performance (1972). He also took the silver medal with Developing Programs for Faculty Evaluation (1974). The bronze went to Ken Doyle for his work on student evaluations, aptly titled Student Evaluation of Instruction (1975). This was followed by John Centra’s Determining Faculty Effectiveness (1979), which synthesized all of the earlier work and proffered guidelines for future practices.

At the end of the decade, one survey found that 55% of liberal arts college deans always used student ratings to assess teaching performance, but only a tiny 10% conducted any research on the quality of their scales. These statistics were reported just in the nick of time before the next blog, which will continue with the flow of research during the 1980s and meta-analyses of previous studies of the “Meso-Meta Era.”

COPYRIGHT © 2010 Ronald A. Berk, LLC

A BerksNotes® GUIDE TO STUDENT RATING SCORE INTERPRETATION: Overview

APPLICATION TO DIFFERENT FORMS
Although each of you is using a different rating form with different numbers of items and scores, those differences do not matter in score interpretation. Whether you’re using a commercial package, such as IDEA, SIR II, PICES, or CIEQ DU SOLEIL, or a “homegrown scale,” there are only so many score reporting possibilities for any form in Likert-type format. So my suggestions are generic and should be applicable to your form. I encourage you to consult the guidelines or manual for your reporting system for more specific information.

FIVE BASIC CATEGORIES OF RESULTS
There are 5 possible categories of results reported for most student rating forms:

1. anchor distribution of percentages
2. item statistics (mean and/or median)
3. subscale statistics (mean and/or median)
4. total scale statistics (mean and/or median)
5. summary of comments to open-ended questions

Your report form may not provide all of the above, but it should certainly give you at least 2 and 4.

WHAT DO FACULTY NEED?
That's a lot of information. You could use all of those results, however, 1 and 2, in particular, provide the most valuable diagnostic info to revise teaching or course materials that will benefit your next course-load of students. These are called formative decisions about teaching. Category 5 can explain the reasons for the ratings to 1 and 2.

WHAT DO ADMINISTRATORS NEED?
Summative decisions about annual contract renewal, merit pay, or promotion and tenure review by department chairs, associate deans, etc. can be based on 3 and 4 and possibly the global item scores.

This blog series will focus primarily on the faculty needs. My next blog will examine the 1st level of interpretation: ANCHOR-WORLD!

COPYRIGHT © 2010 Ronald A. Berk, LLC