Highlights
- Students do not learn more from professors with higher student evaluation of teaching (SET) ratings.
- Previus meta-analyses of SET/learning correlations in multisection studies are not interprettable.
- Re-analyses of previous meta-analyses of multisection studies indicate that SET ratings explain at most 1% of variability in measures of student learning.
- New meta-analyses of multisection studies show that SET ratings are unrelated to student learning.
Abstract
Student evaluation of teaching (SET) ratings are used to evaluate faculty's teaching effectiveness based on a widespread belief that students learn more from highly rated professors. The key evidence cited in support of this belief are meta-analyses of multisection studies showing small-to-moderate correlations between SET ratings and student achievement (e.g., Cohen, 1980, 1981; Feldman, 1989). We re-analyzed previously published meta-analyses of the multisection studies and found that their findings were an artifact of small sample sized studies and publication bias. Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between SET ratings and learning. Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between the SET ratings and learning. These findings suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty's teaching effectiveness.
--Bob Uttl, Carmela A. White, and Daniela Wong Gonzalez
Yeah, *you* know that, and *I* know that, but ain't nobody in administration gonna listen to (psh) *science.*
ReplyDeleteThese things used to be benign in Batshit U, but they're hella malignant now.
ReplyDeleteA private belief among certain instructors is that there may be legal repercussions for some of the most recent policy decisions.
Somebody needs to sue, probably on the basis that relying too much on SETs can lead to gender, race and/or age discrimination (since being unusually attractive and being an easy grader aren't protected classes). One successful lawsuit would do more good than all the studies in the world (though the studies might help the lawsuit).
DeleteFortunately, they're not overemphasized at my institution. The only good thing about institutions that rely on them way too much is that they probably *are* opening themselves up to lawsuits based on the argument that evaluation of teaching leading to hiring and firing decisions should be based on measure that actually, you know, evaluate teaching.