this post was submitted on 23 Apr 2024
216 points (98.6% liked)

Mildly Interesting

17716 readers
674 users here now

This is for strictly mildly interesting material. If it's too interesting, it doesn't belong. If it's not interesting, it doesn't belong.

This is obviously an objective criteria, so the mods are always right. Or maybe mildly right? Ahh.. what do we know?

Just post some stuff and don't spam.

founded 2 years ago
MODERATORS
 

As graders go on grading, their comments become more frustrated and their good-will becomes much sloppier. At least that's the hypothesis to explain this. Researchers found the reverse effect on graders who sorted in reverse-alphabetical order.

all 27 comments
sorted by: hot top controversial new old
[–] bcoffy 84 points 9 months ago* (last edited 9 months ago) (1 children)

I’m a graduate student who does a lot of grading. Canvas gives me an option to: 1. Hide students’ names while grading and 2. sort in order of submission instead of alphabetical order, so I make sure to use both of those options to reduce any biases like that.

[–] [email protected] 40 points 9 months ago (1 children)

Why in order of submission? Why not random?

[–] bcoffy 71 points 9 months ago (1 children)

I only get two options: Alphabetical order and submission order. If I had random I would use it.

[–] kurwa 10 points 9 months ago (1 children)

Do you think order of submission has any bias towards it?

[–] Frozengyro 26 points 9 months ago (1 children)

Yes, you assume the early submitters are on top of it and do better work.

[–] Buddahriffic 20 points 9 months ago

And, even as one who rarely submitted assignments early, IMO it's fair to give the early submissions the advantage of marking fatigue bias. Kinda like a time bonus.

[–] [email protected] 38 points 9 months ago

Semi related: I'm studying for a Linux certification and at the end of each chapter they have 10 practice questions with answers in the back of the book. Almost every time, the explanations of the answers get shorter until there's basically just the answer by question 10. It feels like they just got tired of working

[–] [email protected] 34 points 9 months ago (1 children)

I used to be a teaching assistant at university, and never sorted by name. But based on my experience I don't think it's frustration that accounts for the disparity, it's that as you see more and more assignments you start getting a feel for common issues and are able to point them out more easily. I would always do two passes because of that to ensure that I normalized the weight of my marking.

[–] [email protected] 12 points 9 months ago

I agree with this. It's a bit like the first 2 pancakes, you have to go back over the first half a dozen once you're in the zone.

I used to grade hard copies a lot, after I graded I'd put them in order from best to worst (numerical grades) and then do quick comparisons between an assignment and its neighbours in the pile. It's an easy way to "quality control".

As for the comments, that's a self-discipline issue. If you're giving, say, 4 positives and 4 negatives per assignment and have standard ways of phrasing, it shouldn't deteriorate.

[–] [email protected] 25 points 9 months ago (1 children)

I wonder just how big of a difference your place in the alphabetic order makes in general, because it appears everywhere in life

[–] isles 11 points 9 months ago

It was enough for me to change my last name to "Aaaamazing"

[–] someguy3 23 points 9 months ago* (last edited 9 months ago)

Is this on paper? Who would sort them?

*On a program called Canvas

surnames start with A, B, C, D or E received a 0.3-point higher grade out of 100 possible points than compared to when they were graded randomly. Likewise, students with later-in-the-alphabet surnames received a 0.3-point lower grade—creating a 0.6-point gap.

Random is always the way to go.

[–] [email protected] 15 points 9 months ago (1 children)

So ... graders are fallable humans?

[–] [email protected] 16 points 9 months ago (1 children)

Yea, but this kind of work is needed to encourage blind marking as the default, and not just when standardised testing is involved

[–] [email protected] 3 points 9 months ago (2 children)

I think just randomized order would be enough. It is plausible for teachers to keep track of students' individual progress.

[–] [email protected] 10 points 9 months ago (1 children)

I think blind marking is important. I have literally heard people objecting to proposed grades with phrases like "but he's a bad student" or "but she's really bright."

[–] [email protected] 3 points 9 months ago (1 children)

Unless the assignment is a multiple choice quiz, you can’t really blind it because the thing being evaluated is output from that person.

A million tiny clues will indicate to your subconscious which student’s work you’re grading.

[–] [email protected] 3 points 9 months ago* (last edited 9 months ago)

I can't imagine how, unless you only had 20 of them or something?

Back when I was a TA, I had an average of 120 students per semester and we didn't necessarily grade our own students' work (it was usually divided by topic).

So if I'm grading 120 assignments - or worse, 480 pieces of exam assessment- and only 25% of them are from students I regularly interact with, I don't think my subconscious has any idea 99% of the time.

Even with smaller classes... you're just seeing too many people with similar thoughts and styles over the course of a year for any of it to imprint on your mind that deeply. Occasionally it's going to be obvious, but I still think removing a level of bias through anonymizing is best practice.

[–] [email protected] 1 points 9 months ago (2 children)

So, just the people who get marked last are randomly affected?

[–] [email protected] 2 points 9 months ago (1 children)

Not sure what you mean. Do you think that blind marking would somehow eradicate the bias onto these who get graded later?

[–] [email protected] -2 points 9 months ago (1 children)

No. Exactly the opposite. The problem continues to exist, but now it's hidden.

[–] whostosay 5 points 9 months ago (1 children)

It's improved at least, randomized would be different each time and would influence everyone's grades evenly in a spread out period (in theory.)

[–] [email protected] 2 points 9 months ago* (last edited 9 months ago) (1 children)

Well, yeah. I'd argue that's better than people with certain names being consistently affected.

[–] [email protected] 0 points 9 months ago

They both seem equally bad to me.

You don't have to have either problem though; both can be avoided easily.

[–] [email protected] 2 points 9 months ago

Well I guess I got one less excuse for my bad grades, good thing trades pay well