[
Data investigations This area excites me because I think
it’s a great teaching tool, and it was something I enjoyed doing with my students (as well as asking them to write sample exams, and asking them to improve poor answers). I also think there’s a lot of rich data from the technique, and there are a lot of small changes that could be explored for their impact, some of which would adapt a generic model to be closer to the kind of calibrated peer assessment at UCLA (but without buying into the particular product), for example: 1. Around the benchmark, it would be interesting to know: 1. if students who perform poorly, also perform poorly in the assessment 2. if students who perform poorly, in fact perform poorly as assessors too 2. To reduce the need for moderation and (hopefully) increase quality, it would be interesting to look at some reliability analysis of student ratings, and whether or not targeting discrepancies (or, tracking students who tended to disagree) could reduce variance in grades, and perhaps also support those students in their assessments and assignments (per ‘1’) 3. Analysis of the written feedback would be interesting. In my work I asked students to suggest 3 improvements that could be made, using a drop down menu to categorise those improvements according to the rubric facets they were assessed on. Looking at the kinds of suggestions made is much easier with that kind of semantic annotation, but even just taking the written feedback and analysing the topics/themes should provide useful insight. It would also be interesting to explore whether ‘poorer’ raters also gave poorer written feedback or not (and especially if not, why is that the case?) Update I’d forgotten, I actually posted a short powerpoint of the model I use, along with some teaching resources in an [earlier post on this issue!]4
Footnotes
-
/static/2015/11/0d41b348606fd6e4959ee5aa_150_marking.jpg ↩
-
http://sjgknight.com/finding-knowledge/2015/07/diagnostic-peer-assessment/ ↩