Bias and Trends in Student Evaluations in Online Higher Education Settings
Main Article Content
Abstract
End-of course evaluations have been frequently used to assess teaching effectiveness and influence critical decisions about faculty contract renewal, future course assignment, tenure and promotion in higher education. This quantitative study sought to determine whether there are differences in student perceptions of faculty performance based on gender or faculty status (full-time vs. adjunct) in an online higher education environment. It also sought to answer these questions: 1) Do adjunct faculty tend to grade more leniently than full time faculty, and as such, do adjunct faculty receive higher evaluation ratings than full time faculty, who may be more stringent in grading? 2) Do student evaluation scores differ depending on the course being evaluated?  3) Does gender or faculty status impact student response rates? Survey responses from a total of 683 sections associated with 24 courses were analyzed from the March 2018 to January 2019 timeframe. Due to the broad range of class sizes and differences between faculty characteristics, the variances for each comparison sample were observed to be significantly different using Levene’s test for equal variances. Thus the Mann-Whitney test for two variables and the Kruskal-Wallis test for evaluation of significant difference between more than two variables were used on the data. While other literature and personal anecdotes may indicate that gender bias exists, this study did not indicate that gender bias is occurring in online higher education courses taught for the time period studied, suggesting gender neutrality.
Article Details
References
Adams, M. J., & Umbach, P. D. (2012). Nonresponse and online student evaluations of teaching: Understanding the influence of salience, fatigue, and academic environments. Research in Higher Education, 53(5), 576-591.
Ancell, K., & Wu, E. (2017). Teaching, learning, and achievement: Are course evaluations valid measures of instructional quality at the University of Oregon? Retrieved from https://provost.uoregon.edu/files/course_evaluations_wu_ancell.pdf
Andersen, K., & Miller, E. D. (1997). Gender and student evaluations of teaching. PS: Political Science and Politics, 30(2), 216-219. doi:10.2307/420499
Andrade, C. (2019). Multiple testing and protection against a type 1 (false positive) error using the Bonferroni and Hochberg corrections. Indian Journal of Psychological Medicine, 41(1), 99-100. doi:10.4103/IJPSYM.IJPSYM_499_18
Armstrong, R. A. (2014). When to use the Bonferroni correction. Ophthalmic and Physiological Optics, 34(5), 502-508. doi:10.1111/opo.12131
Avery, R. J., Bryant, W. K., Mathios, A., Kang, H., & Bell, D. (2006). Electronic SETs: Does an online delivery system influence student evaluations? Journal of Economic Education, 37, 21–37.
Bacon, D. R., Johnson, C. J., & Stewart, K. A. (2016). Nonresponse bias in student evaluations of teaching. Marketing Education Review, 26(2), 93-104. doi:10.1080/10528008.2016.1166442
Basow, S. A., Phelan, J. E., & Capotosto, L. (2006). Gender patterns in college students’ choices of their best and worst professors. Psychology of Women Quarterly 30(1), 25–35.
Beran, T., & Violato, C. (2005). Ratings of university teacher instruction: How much do student and course characteristics really matter? Assessment & Evaluation in Higher Education, 30(6), 593-601.
Black, D., Bissessar, C., & Boolaky, M. (2019). Online education as an opportunity equalizer: The changing canvas of online education. Interchange, 50, 423-443. doi:10.1007/s10780-019-09358-0
Boring, A. (2015). Working paper: Gender biases in student evaluations of teaching. Retrieved from https://www.ofce.sciences-po.fr/pdf/dtravail/WP2015-13.pdf
Boring, A. (2017). Gender biases in student evaluations of teaching. Journal of Public Economics, 145, 27-41. doi:10.1016/j.jpubeco.2016.11.006
Boring, A., Ottoboni, K., & Stark, P. B. (2016a). Student evaluations of teaching (mostly) do not measure teaching effectiveness. ScienceOpen Research. doi: 10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1
Boring, A., Ottoboni, K., & Stark, P. B. (2016b). Student evaluations of teaching are not only unreliable, they are significantly biased against female instructors [Blog post]. Retrieved from https://blogs.lse.ac.uk/impactofsocialsciences/2016/02/04/student-evaluations-of-teaching-gender-bias/
Braga, M., Paccagnella, M., & Pellizzari, M. (2014). Evaluating students’ evaluations of professors. Economics of Education Review, 41, 71-88. doi:10.1016/j.econedurev.2014.04.002
Carrell, S. E., & West, J. E. (2010). Does professor quality matter?: Evidence from random assignment of students to professors. Journal of Political Economy, 118(3), 409-432. doi:10.1086/653808
Cavanaugh, J. K. (2006). What did you get? A faculty grade comparison. Quality Assurance in Education: An International Perspective, 14(2), 179-186.
Chapman, D. D., & Joines, J.A. (2017). Strategies for increasing response rates for online end-of-course evaluations. International Journal of Teaching and Learning in Higher Education, 29(1), 47-60.
Crumbley, D. L., & Reichelt, K. J. (2009). Teaching effectiveness, impression management, and dysfunctional behavior: Student evaluation of teaching control data. Quality Assurance in Education, 17(4), 377–392.
Doerer, K. (2019). Colleges are getting smarter about student evaluations. Here’s how. The Chronicle of Higher Education, 65(18), A8.
Eagly, A., & Karau, S. J. (2002). Role congruity theory of prejudice toward female leaders. Psychological Review, 109(3), 573–598.
Fan Y., Shepherd, L. J., Slavich, E., Waters, D., Stone, M., Abel, R., & Johnston, E. L. (2019) Gender and cultural bias in student evaluations: Why representation matters. PLoS ONE 14(2): e0209749. https://doi.org/10.1371/journal.pone.0209749
Groves, R. M., & Couper, M. P. (1998). Nonresponse in household interview surveys. New York: Wiley
Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. Hoboken, NJ: Wiley.
Isely, P., & Singh, H. (2005). Do higher grades lead to favorable student evaluations? Journal of Economic Education, 36(1), 29–42.
Johnson, V. (2003). Grade inflation: A crisis in college education. New York: Springer
Kezim, B., Pariseau, S. E., & Quinn, F. (2005). Is grade inflation related to faculty status? Journal of Education for Business, 80(6), 358-363.
Kierstead, D., D’Agostino, P., & Dill, H. (1988). Sex role stereotyping of college professors: Bias in students’ ratings of instructors. Journal of Educational Psychology, 80(3), 342–344.
Krautmann, A. C., & Sander, W. (1999). Grades and student evaluations of teachers. Economics of Education Review, 18(1), 59-63. doi:10.1016/S0272-7757(98)00004-1
Lederman, D. (2018, November 7). Online education ascends. Inside Higher Ed. Retrieved from https://www.insidehighered.com/digital-learning/article/2018/11/07/new-data-online-enrollments-grow-and-share-overall-enrollment
Linse, A. R. (2017). Interpreting and using student ratings data: Guidance for faculty serving as administrators and on evaluation committees. Studies in Educational Evaluation, 54, 94-106.
Lippmann, S., Bulanda, R. E., & Wagenaar, T. C. (2009). Student entitlement. College Teaching, 57(4), 197-204.
MacNell, L., Driscoll, A., & Hunt, A. N. (2015). What's in a name: Exposing gender bias in student ratings of teaching. Innovative Higher Education, 40(4), 291. doi:10.1007/s10755-014-9313-4
Mengel, F., Sauermann, J., Zölitz, U. (2019). Gender bias in teaching evaluations. Journal of the European Economic Association, 17(2), 535-566. doi:10.1093/jeea/jvx057
Mitchell, K., & Martin, J. (2018). Gender bias in student evaluations. Political Science & Politics, 51(3), 648-652. doi:10.1017/S104909651800001X
Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? Assessment & Evaluation in Higher Education, 33(3), 301-314.
Reynolds, D. (2015). Variability of passing grades in undergraduate nursing education programs in New York State. Nursing Education Perspectives, 36(4), 232-236. doi:10.5480/13-1235
Rosen, A. S. (2017). Correlations, trends and potential biases among publicly accessible web-based student evaluations of teaching: A large-scale study of RateMyProfessors.com data. Assessment & Evaluation in Higher Education, 43(1), 31-14. doi:10.1080/02602938.2016.1276155
Ryerson University v. Ryerson Faculty Association, CanLII 58446 (2018)
Sax, L. J., Gilmartin, S. K., & Bryant, A. N. (2003). Assessing response rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44(4), 409–432.
Sidanius, J. & Crane, M. (1989). Job evaluation and gender: The case of university faculty. Journal of Applied Social Psychology, 19, 174-97.
Sonner, B. S. (2000). “A†is for ‘Adjunct’: Examining grade inflation in higher education. Journal of Education for Business, 76(1), 5-8.
Stroebe, W. (2016). Why good teaching evaluations may reward bad teaching: On grade inflation and other unintended consequences of student evaluations. Perspectives on Psychological Science, 11(6), 800-816. doi:10.1177/1745691616650284
Statham, A. Richardson, L., & Cook, J. A. (1991). Gender and university teaching: A negotiated difference. Albany, NY: State University of New York Press.
Uttl, B. & Smibert, D. (2017). Student evaluations of teaching: teaching quantitative courses can be hazardous to one’s career. PeerJ, 5, e3299.
Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22-42. doi:http://dx.doi.org/10.1016/j.stueduc.2016.08.007
University of Oregon Office of the Provost. (2019, March). Revising UO’s teaching evaluations. Retrieved from https://provost.uoregon.edu/revising-uos-teaching-evaluations
University of Southern California Academic Senate. (2018, September 20). Teaching evaluations update. Retrieved from https://academicsenate.usc.edu/teaching-evaluations-update/
Weinberg, B. A., Hashimoto, M., & Fleisher, B. M. (2009). Evaluating teaching in higher education. The Journal of Economic Education, 40(3), 227-261. doi:10.3200/JECE.40.3.227-261