Scantron and Modern MCQs

My professional society, the Casualty Actuarial Society, went all-digital in response to covid. It was actually the second attempt to do so – the first attempt (a few years before covid) had, to be kind, some administrative problems. But the covid-era test, administered at testing centers, went very smoothly, and both the candidates and the graders felt it was an improvement.

There are still some multiple choice questions, especially on the earlier exams. I expect some of those to be replaced with “templates” that pop in various numbers, at least for the numeric questions. There are still some fact-based questions (do you know the standard provisions of a US personal auto policy) that will likely remain MC.

The later exams are all “open answer”, and the candidate is presented with a spreadsheet environment (very similar to excel) to do their work. Some of the questions are more “essay” in nature and the students just type sentences into the spreadsheet.

The graders interact with the answers on-line. They don’t have to puzzle out bad handwriting. They can do (unsaved) scratch work in the spreadsheet to try to reproduce what the candidate did, if that’s unclear. They don’t have to wait for the home office to photocopy all the paper answer pages and mail them out.

It’s awkward for drawing graphs, but otherwise it works very well.

I don’t know if my ballot is read on an official “Scantron” brand machine, but it sure looks like the “dot tests” I had as a kid.

You also have to deal with them during grading, where you need the graders to see the answers without seeing the identifying information, but still attach the scores to the right student, and potentially have multiple graders checking the same answer for consistency. That’s a lot easier in an electronic format.

Oh, and the move to digital tests has, in some cases, also resulted in changes in technology allowed. On Ohio’s state math tests, for instance, the test itself has Desmos (a very popular and powerful online graphing calculator) built into it, and students are allowed and expected to use it. It’s so much more capable and easy-to-use than the TI-84 (which hasn’t changed significantly for decades) that it’s no comparison: The only argument for handheld calculators at all any more is that that’s what you’ll be allowed to use on the big tests, and if the big tests move away from that, they’ll be completely obsolete.

Honestly, as someone pretty deeply involved an AP scoring, that’s not a huge problem. But we do scan all resppnses in these days to score on a computer, and yes, the scanning is hugely expensive, and when it doesn’t get done on time, its double triple quadruple more hugely expensive.

Oh, sure, there are ways to do it on paper. It’s just a lot easier on a computer. Which can also have the rubric built in. Like, on a calculus FRQ, there will be 9 points total, with a breakdown like “Did the student write the integral correctly? 1 point. Did the student attempt to use integration by parts to evaluate the integral? 1 point. Did they solve the integral correctly? 1 point”. I’ve never done the grading myself, but I imagine that the screen has a checkbox for each of those points.