Can someone explain the A Level and GCSE scandal in the U.K.?

I think that all makes total sense and, as you say, totally predictable.

I’ve been told the system in Ireland allowed teachers to basically flag up a student as exceptional, to avoid the situation that’s really caused the most outcry- the excellent student at a school which normally has poor results getting downgraded- as well as allowing teachers to rank multiple students at the same level if they really are too close to rank in order. But yes, they’re expecting similar issues, especially as they’re not used to officially predicting grades (this from an Irish friend, who had it from teacher family, no cites).

Grade inflation is not the same as the teacher ‘not knowing’ what the level of their students is. I acknowledge that they are different things.

This makes a lot of sense, and raises a much bigger question about the value of exams as a means of assessing students. Is this student an actual A student or did they get lucky on the day? Is this one really a C or was the pollen count really high that day? But this takes into murky waters.

For those interested in the fall-out: some universities are taking on bigger intakes (and facing financial troubles for doing so); some are taking on smaller intakes (and facing financial troubles for doing so); the Education Minister is not resigning but apparently offered to and was told to stay in post. Cynically, one might think that’s because the government would rather brazen this out than admit it might have made a mistake. However, pretty much every newspaper is gunning for him so keeping him might drag this out for longer.

The UK government seems to have the attitude that you should just ride any storm out because the media will have moved onto the next thing within a few weeks.

To be fair, all the evidence says that they’re right.

ETA: E.g. the latest poll, at the height of colossal yet avoidable fuck-up, has the Conservatives beating Labour 41-37 in voting intention. That’s a big chunk of the country looking at this shambles (plus the highest in Europe COVID death toll) and saying, “Yup, these are the guys I want in charge”. Big scandal, small fallout.

I guess a fair amount of that is from anti-Corbyn hangover - has Starmer really had a chance to set out his policies/vision for the country? I’m not a fan of the current government but there is still a suspicion that Corbyn might have been worse (impossible to say either way of course). I have never voted Labour in my life, for various reasons - my first GE was 2005, and I didn’t like Blair even before Iraq, certainly didn’t want to vote for him after. Ditto Brown. I was one of the few who was reasonably satisfied with the coalition government, of course they still did bad things (don’t all governments?) but despite the tuition fees issue I feel the LibDems did a reasonable job of moderating the Tories wilder tendencies. I voted Tory in 2015 because I wanted a referendum, in 2017 I voted for a local independent candidate as I didn’t really believe in any of the others, and in 2019 I voted LibDem as the most representative of my views. For those last three I have been in one of the safest Tory seats in the country so haven’t had much effect anyway. At the moment I’d probably look to vote more based on local personalities than national parties but Starmer does appear to have a decent chance of getting my vote, we will see.

Hmm! Most of my acquaintances are of the opinion that whichever party had been in charge back in the Spring, it would still have been a disaster. Even some die-hard Labour supporters think that with Corbyn in No10 it would have been even worse.

I think that the minister should go - he saw what happened in Scotland and swift action then would have saved all this heartache.

Even a 10 year-old would have known to check if they were repeating the problems seen in Scotland - so you have to assume they did that. But it takes a bit more bravery to examine them, see that there were big issues ahead, strike out all the work of the algorithm (that Williamson probably signed off on), and then accept the new problems that would arise from doing that. However that’s what should have happened. Easy to say in hindsight though.

I think these two points pretty much cover it.

A more in depth analysis of the algorithm and its failings is here:
http://thaines.com/post/alevels2020

One of the many occasions in a case like this when I’m mad there isn’t any discussion of the technical details in the mainstream press. I wouldn’t something that in-depth but I haven’t seen any details at all of the algorithm itself anywhere in the press.

Also as someone who does some pretty intense number crunching, in their day job I 100% agree that its crazy that they claim to be limited by “computational problems”. That problem is a tiny fraction of the computation done by a AAA computer game in a 16ms frame.

That may be true. But, if it is, at least part of the reason for it is the systematica hollowing-out of state capacity and resources that has gone on under the last ten years of Tory-imposed austerity. The UK state has been seriously degraded by the policies adopted in response to the GFC in a way that didn’t happen in comparable European countries, and that was a policy choice by the party in power throughout this period.

There’s an interesting human element to this fuck up.

People were given a job. They were told:

Here are individual level teacher assessments of pupils. Here are teacher rankings of pupils. Here are previous A-level results for each class within each school for the past 4 years. Please use this data to create an accurate assignment of A-level grades to these pupils as there is no exam this year and we still need to hand out grades. Do not just use the teacher assessments as we think these are likely to be wrong.

This is at first glance a very difficult problem, and an important one, and there isn’t all that much time. So people start trying really hard to solve it. But it’s not a difficult problem. It’s an impossible one. You cannot create accurate individual-level output data from class-level input data. The information you need is not there. It cannot be done, as a matter of basic information theory.

So why didn’t anyone (involved in the process) notice and/or say this? I’ve done the same myself - worked really hard at a problem and come up with some complex non-solution, rather than say from the beginning, “You know what, this will never work, we need to face that fact and act accordingly.” It’s a mix of pressure, and desire to solve the problem, and a kind of groupthink.

I don’t want to let anyone off easy. There were people (teachers, the Education Select Committee, parents) who spotted the problem and weren’t listened to. But I’m interested in the processes in the room that prevented anyone from stepping back and clearly stating the problem.

As I understand it, Gavin Williamson told Ofcom very strongly that he didn’t want grade inflation. Everything was based on that.

The current result… massive grade inflation.

[Moderating]

A reminder, everyone, that this thread is in General Questions. A discussion of what the problem is is appropriate for this forum. A discussion of various other solutions that might have been applied to the problem is a bit of a digression, but still acceptable. A discussion of who’s to blame for the problem, however, is decidedly political, and is better suited for the Politics and Elections forum.

This problem is not confined to A levels or GCSEs, it also covers a lot of proffessional qualifications and vocational qualifications such as BTEC.

If you thought it was as bad as it can get, you’d be wrong because the academic side of things has been relatively straightforward to develop an imperfect solution.

Many proffessional and vocational courses also operate at advanced degree level but are often modular - so you take a number of core units and a number of specialist and optional units.

Some of those units are completely based on coursework - no surprise for all the welders, electrical installers, gas fitters etc. Unless there is some output then there is nothing to evaluate skill levels and no assessement of competence is possible.That could make the learner effectively unemployable - especially on courses that are required prior to obtaining a license for swuch work - such as pressure vessel welding or electrical installation, working at height or handling of hzardous materials.

Some units are a mix of coursework and theory and lots of units have semester based exams - so you’d complete the first part of a unit in semester 1 and the second part in semester 2.

The point is that absolutely no one solution will work for these types of qualifications, some of the units are essential foundation to further studies so you cannot - for example - fudge math assessment because if you do not have the mathematical tools then you simply will not be able to complete your studies in other aspects of the course - such fundamental units must be assessed and competence determined before further enrollment.

Yeah, this is a big issue. Latest I heard is that Pearson - who run the BTEC quals - are pulling them (very last minute) to regrade in line with A-levels and GSCEs. I don’t know if it’s the article or the situation that’s confused, but I found it very hard to work out from the link exactly what the plan is.

Following up on some of the points above, here’s a timeline of who knew what and who warned what, and when

Wandering into Great Debates territory, I’m wondering if we shouldn’t establish in data protection law the principle of full openness/transparency of any automated decision-making that has such a powerful effect on people’s lives. The potential for such effects once AI really takes off is quite alarming.

Wouldn’t the teacher predicted grades be a matter of record going back years? Seems like that would be useful data in arriving at a fair grade - we basically know how accurate each school is at predicting grades (each actual teacher in an ideal world though I guess that data doesn’t exist) we know how these predictions vary across subject and the grade scale, so we can accurately apply teacher assessments with an appropriate modifier.

I guess schools go through a lot of change these days so there may not be value in the historic prediction performance in many cases. Water under the bridge now in any case.