Based on your Olympic scoring observations and yesterday’s blog, what does any of that have to do with education? NOTHING! Kidding. Here are a few thoughts.
EDUCATIONAL IMPLICATIONS
The process for improving the objectivity of scoring and grading in classroom assessment, evaluation of teaching performance, and program evaluation has been fraught with many of the same challenges, just maybe not a cauldron, but certainly a large bucket.
Student Assessment. The methods have ranged from multiple-choice tests to essay tests to performance tests to student portfolios (to OSCEs with standardized patients in medicine and nursing). The increased precision of the judgmental scoring systems with explicit rubrics has reduced bias and improved the validity and reliability of the scores to measure learning outcomes.
Faculty Evaluation. Among 14 potential sources of evidence to evaluate faculty teaching, most ALL are based on the judgments of students and “informed” professionals. There are more than 10 possible types of bias that can affect those judgments. However, the triangulation of multiple sources can compensate to some degree for the fallibility of each measure. This approach can also be generalized to promotion and tenure reviews based on the faculty portfolio.
Program Evaluation. Any of the preceding measures plus a collection of other quantitative and qualitative sources are used to determine program effectiveness based on specific program outcomes. Tests, scales, questionnaires, and interview schedules provide complementary evidence on impact.
STATE-OF-THE-ART
Unfortunately, let’s face it: Complete measurement objectivity has eluded educators since Gaul was divided into 4.5 parts. It’s an intractable problem in evaluating faculty, students, and programs. Psychometrically, the state-of-the-art of behavioral and social assessment indicates it is just as fallible as Olympic scoring, but it’s the best that we have.
If we could just balance these scoring systems with a speed-teaching, speed-learning, or speed-mentoring, time-based approach, we could come closer to what we observed in the Olympic events. Also, there may be a lower risk of injury and wipe-outs in the classroom.
What do you think? Any ideas on any of the above are welcome. If you come up with the solution, you could receive a medal! However, I’m not sure of the scoring rubric for that medal, but I know it will be fair.
COPYRIGHT © 2010 Ronald A. Berk, LLC
I was thinking about this a lot during the Olympics. I agree that even if there is a timer involved, it is still a judgment of some sort. I suspect that the striving for objectivity comes from wanting to determine who the best athlete is, but the judgments are de facto about determining the best _performance_. Because of that, they should not try to be 'objective' at the expense of aesthetics; the judgment of sports should not ignore or devalue creativity and innovation. I think educators are faced with a similar conundrum in giving grades. Similarly, in testing only the memorization of the right 'facts,' the valuation of thinking critically and creatively suffers.
ReplyDeleteIf you're interested, I commented on the role of objectivity in sport, particularly figure skating here:
http://thegrandperspective.blogspot.com/2010/02/where-have-all-flowers-gone.html
thanks for your thoughtful posts!