Bias in grading

From this document on approaches to learning, consider the following:

The parent indicated how frequently the child exhibited the following behaviors or characteristics. The response scale included four points ranging from "1 = never" to "4 = very often."

This subscale is composed of the following items:

10. Keep working at something until {he/she} is finished?

13. Show interest in a variety of things?

15. Concentrate on a task and ignore distractions?

18. Help with chores?

22. Eager to learn new things?

24. Creative in work or in play?

The above factors, averaged together, form (if I have not misread the documentation) the Approaches To Learning scale.

I'm not sure that item 18 really belongs here. It makes life much nicer for teachers, but says nothing whatever about how well someone will learn.

And items 10 and 15 don't address multitasking. There are times when I bear down, get in the zone, and do one task until it is done. There are others in which I pump until the well runs dry, switch to another task to refresh for a while, and come back to the original one later. But they sure make a teacher's life easier.

So one of the items doesn't deal directly with learning, and two others only partly do--but all three of them make a good impression on the teacher.

OK, look at a different set; these the teacher is supposed to evaluate:

11. Keeps belongings organized.

14. Shows eagerness to learn new things.

15. Works independently.

21. Easily adapts to changes in routine.

23. Persists in completing tasks.

24. Pays attention well.

In third and fifth grade, the following item was added to the SRS and added to the Approaches to Learning subscale:
26. Following classroom rules.

Here too: 21 is a problem. If I'm in the zone, I don't want distractions thank you very much, school bell or no. And if I'm multitasking--I mentioned that before. And I have somewhat different ideas about what constitutes "organized." Granted, disorganization can rise to a level that seriously interferes with getting things done, but below that it isn't obvious that it matters much. And 15: I've taught students who almost refused to work independently, but who mastered the subject just fine. And do I need to mention #26?

Once again the list includes things that make the teacher's life easier but don't necessarily effect learning.

This may seem a bit banal, but it turns out to make a difference.

A longitudinal study (I didn't see anything obviously wrong with it, BTW) of K-5 grade students found that teachers, unaware of their students' standardized test scores, assigned grades to boys significantly lower than those they gave girls with the same test scores. This was true in every case where there were adequate statistics (black 5'th graders were lower statistics--more had moved out of the study areas).

Girls were better than boys at reading skills on tests, but the spread in grades was bigger. Boys were slightly better at math tests, but their grades were worse; similarly with science.

Grades did not reflect mastery.

Since grades are a large factor in admission to higher education, this can have fairly far-reaching consequences.

So, why did the (mostly female) teachers grade girls higher than boys?

The researchers found that if they included the ATL score in their analysis, the differences almost vanished; in fact sometimes boys were graded slightly above the girls. Girls had significantly higher ATL scores than boys. (To try to take into account any bias due to the teacher producing both grades and ATL scores, they used ATL scores from 2 years earlier, presumably from a different teacher.)

The obvious conclusion is that teachers were including social factors in their grading.

That seems inappropriate. You can argue that these social factors will contribute to better performance in academic settings later, but AFAIK that claim is unproven, and in any event doesn't address the problem that the grade is not a clear measure of mastery. Splitting out attitudes from academic skills gives more information than blending them.

And as noticed earlier, those attitudes combine factors relevant to learning and factors relevant to class maintenance.

So, what can we do about this? Let's assume this is verified by other work.

• The default approach is to circle the wagons, defend the institutions, pretend nothing is wrong and that mastery really entails social skills and we should leave the experts alone. This is probably what will happen.
  
• We could try to ask teachers to split out ATL sorts of things in the hope that they'd create academic grades that better reflected mastery. I don't believe this will work at all.
  
• We could try to find some new (or old) approaches to teaching that take squirrely boys into account better. This isn't easy to do in an industrial setting like a classroom. Home schoolers have an overwhelming advantage here.
  
• We could have same-sex classes. I keep hearing rumors that these have academic advantages and I suppose I should poke around and find out the facts sometime. But I can hear the screams now. This one is political ebola.
  
• We can use more and more standardized tests. I'm not thrilled about this option. It invites cheating (as seen already), teaching to the test, and generally locking everybody into the same one-size fits all mold.
  

I wonder if this happens with male teachers? Not that there are very many of them at that level. And I wonder if it carries over into high school and college?