At the risk of raining on the DOWN WITH EVALS parade

While I can get as knee-jerkingly DOWN WITH COURSE EVALUATIONS as the next professor, an article that cropped up in my Twitter feed (retweeted from the New Faculty Majority) prompted my poor knee to jerk in the opposite direction.  Tracy Vaillancourt, a professor at the University of Ottawa, ran an experiment that proved "that students who received poor grades were 10 to 19 times more likely to make negative comments about their evaluator than those who received high grades."  Ah-ha! Validation! What we cranky dragons have always known! (Certainly, most of us have already figured out that "'once a poor grade is received, the feedback is all but ignored and the focus is on the grade.'")  But wait, hang on:

To test her hypothesis that poor teaching evaluations are a form of revenge, Vaillancourt designed three related studies, each involving between 150 and 176 mostly first-year university students.

Students were given up to 20 minutes to write a short essay on an assigned topic, and were told their work would be graded by professors or teaching assistants. The essays were then randomly assigned a high or low grade and, in some cases, comments such as “No suggestions, great essay!” or “This is one of the worst essays I have ever read!” [my emphasis]

Students were then asked to provide feedback to their evaluator about his or her marking ability, fairness, helpfulness and general competence.

In the first study, participants given poor marks and negative comments were almost 10 times more likely to rate their evaluator harshly than those given high marks and positive comments.

It also found that students’ level of self-esteem influenced their ratings. Among those whose essays received low marks and negative comment, high self-esteem was associated with the lowest evaluations. Conversely, those with high self-esteem who received positive feedback from their evaluators rated them the highest.

The grades were "randomly assigned"? It's not as though students are robots, spewing out papers and then mechanically processing the grades.  Randomly grading (really, "grading") the students inflates something, all right--it inflates the likelihood that students in the survey received grades wildly off the mark of their own self-assessment, for no obvious good reason.   Sure, there will be students who respond with joyful cheers and popping champagne bottles to that unmerited A, but what about the student who received an F and knows perfectly well that s/he didn't earn one?  Students are perfectly capable of grasping the difference between tough grading and grading by chucking papers down the proverbial stairs.  Moreover, a comment like "this is one of the worst essays I've ever read" is...suboptimal, to say the least; when I deal with truly disastrous prose, I at least try to preface my comments with "I'm afraid..." As presented in the write-up, the negative comments seem designed to exacerbate the negativity and downplay the helpfulness.  In other words, the methodology seems designed to inflate outcomes at both ends of the scale: happy students (woo-hoo! awesome instructor is awesome!) and annoyed students (death by ballpoint pen!), with some (or many? we don't know) justifiably infuriated. 

Comments

[Contingent Cassandra]

That does seem like a not particularly valid, not to mention ethically questionable (since it wastes the students' time and perhaps reinforces the idea that teachers grade randomly) experiment. After all, anybody at pretty much any university who wants to investigate whether or not grades, other forms of teacher feedback, or both correlate with student eval scores in freshman comp (or any core course) has reams and reams of data potentially available to hir. It might take filling out some human subjects research forms, and figuring out some logistics, but the experiment you describe would have required that, too. I don't understand what the researchers thought they would accomplish by setting up the experiment the way they did.

[Brandon]

That is definitely weird. What we really seem to learn from this study is that you are more likely to get negative evaluations if you grade students in way that makes negative evaluations more reasonable.