Exams 2020 and the demise of Ofqual, who pays the ferryman? NEW UPDATE

NEXT UPDATE

20th August 2020

Today brought a further twist in the intrigue surrounding the decision by the government to press on with the flawed Ofqual algorithm for calculating the public examination results this summer. Doubts have been raised about what the Secretary of State and Schools Minister knew well in advance of deciding to revert to teacher Centre Assessed Grades or CAGs. Doubts are creeping in about the underlying motivation behind maintaining the existing inequalities in the system.

The Guardian has revealed two things that cast considerable doubt on the assertions that the ministers involved, Gavin Williamson and Nick Gibb, were not aware of the Ofqual algorithm 'problem' until the weekend. Yesterday, the Guardian revealed that 'Ofqual ignored exams warning a month ago amid ministers' pressure'. It seems that whistle blowers are  heading down the well worn path of breaking ranks in the face of what appears to be disingenuous claims by minsiters. Unnamed 'external advisers' were reported as warning of the methods used being unreliable and essentially "volatile".They confirm that Schools Minister, Nick Gibb, "as having piled on pressure to keep grade inflation down".

Then, later this morning they reported that the same schools   'Minister admits he was warned about concerns over exams algorithm'. This is very concerning and points to a deliberate adherence to a plan that had been discredited much earlier. The Times quickly pitched in with,  'Gavin Williamson was warned about risk of exam fiasco' since a former director general at the Department for Education (DfE), Jon Cole, had written with concerns about the algorithm used over a month ago.

If 'volatile" was the word of the day, incendiary might be another to add to the list. This is because the Guardian had earlier told us that one of the Ofqual advisers was Public First,  a PR 'Firm linked to Gove and Cummings hired to work with Ofqual on A-levels'.  Run by James Frayne and Rachel Wolf, the links to Gove and Cummings, and their political and ideological objectives, is not in doubt. This is the equivalent of adding petrol to the fire that will now consume Ofqual. With the flames fanned by the Guardian, who reported late yesterday that 'Ofqual exam results algorithm was unlawful, says Labour', it raises more questions about the legal advice received by the DfE and Ofqual. It doesn't take a 'rocket surgeon' to work out where all this is travelling.

Wilful or stupid?

If Public First are advising on  communications with the public, they are not very effective. If the government department involved is secretive in its deliberations, then this is the first mistake. Compounding this with spin is the second. 

Yesterday, Schools Week reported that they had used an FOI to extract records from the DfE in 'The DfE has published its first board minutes in over 2 years – here’s what we learnt'.  They are quite right in their conclusion that the Board of the DfE failed to discuss the examinations arrangements.  With Williamson in the chair, alongside, Gibb and Donelan, it is astounding that this was not on the agenda or discussed.  It is also odd that the possible risk, and need for contingencies,  was not part of the discussion at the previous meeting on the 4th February 2020. Although 'risk appetite' was on the agenda, it seems the advance of COVID-19 was not seen as a risk. The backlog of minutes are now available on the DfE www site.

The jury is still be out on the idea of this behaviour being either 'stupid or wilful' (see TEFS 11th August 2020 'Stupid or wilful: Exam results common sense fails to spread from Scotland'). After all, Scotland, along with the other jurisdictions in the UK, set out with protecting standards as their top priority. The way they did this had differences and evolved in parallel. On another level, the approach would reinforce the wide social divisions that already exist in the system across the whole of the UK (see TEFS 7th August 2020 'Qfqual builds a concrete wall').

Yet there remains a niggling doubt that the 'genetic determinism' in intelligence and attainment promoted by Cummings means that the system had to be protected to further his undoubted aims. In February last year, Cummings posted a blog ‘Genetics, genomics, predictions & ‘the Gretzky game’ — a chance for Britain to help the world’ (See also TEFS 10th January 2020 ‘Genetics, Intelligence, Social Mobility and Chinese Whispers’ for a detailed appraisal). In it he describes the rationale behind genetic testing and its role in both health and education. The recent widespread roll-out of new technology for Coronavirus testing is deploying sophisticated instrumentation more in line with the goal of comprehensive gene testing of the wider population. In 'Coronavirus testing with a sledgehammer designed for universal genetic sequencing' on 3rd August 2020, TEFS outlined the potential dangers of using a "sledge hammer to crack a nut" and the consequences of "The Cummings agenda".

Could Cummings be the mythical ferry man carrying Ofqual to its final resting place? but the question is who is paying the ferryman?

What next?

One can only expect further revelations to emerge in time. Ofqual have confirmed receipt of my FOI request for the Ofqual Board minutes not released since September 2019. They have until 16th September 2020 to comply, but it would be a simple act for them to release the minutes now and save everyone the wait. 

These problems will not go away and they have tainted trust and confidence in how examinations are conducted. The fact that BTEC results had to be pulled  by Pearson late last night is an indication of how bad the oversight and planning has been. Pearson had little choice but to start again with their calculations.

A full inquiry capturing the methods and motivations across the whole of the UK is needed before the inevitable moves to reform the examinations systems takes place. 

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.

EARLIER POSTS

UPDATE 19th August 2020

FOI request.

A Freedom of Information request has been made today to Ofqual regarding the minutes of meetings of their Board. There are no minutes posted on their www site since September 2019. On past records there would have had around seven meetings since. Also, the crisis means other meetings between Ofqual officials and the Board, including their Standards Advisory Group, were likely. It is assumed they also considered legal advice about their process. This would have included equality issues that were surely foreseen as likely to happen. Today, the Guardian with 'Ofqual exam results algorithm was unlawful, says Labour' indicated that the shadow Lord Chancellor thought "ministers and Ofqual would have been aware of at least three breaches of the law in the standardisation formula used". If that is the case, the the Board must have had some indication of this and gave a response. It is usual for the minutes to be published regularly, so a cessation of this openness since September of 2019 seems puzzling.

Original post 18th August 2020.

It seems like the autopsy has started before Ofqual the victim has died. It's like an episode of NCIS with Gibbs and his colleagues chasing down the ‘perp’ with the medical  expert, played by David McCallum, investigating the cause of death. The Education Secretary, Gavin Williams, apologised earlier today to back up an apology from Ofqual through the Chair of its Board, Roger Taylor. Now the Chair of the Commons Education Committee has called for Ofqual to be put down and laid to rest in the Department of Education (DfE). But what will replace it? A private provider is perhaps a good bet. They would have a lot to gain from using cheap labour, inflated costs and charging for appeals. Then the question is not one of blame but becomes ‘who will pay the ferryman?’ 

Blame for the examinations debacle is falling more upon Ofqual today than on the political decisions made early on that determined the way they conducted the calculation of results. Many observers will be confused about who was in charge as the main suspects point to each other and say ‘it wasn’t me gov’, it was him.’

The chair of the Ofqual Board, Roger Taylor was sent out to apologise yesterday and seemed very nervous indeed. His statement is posted here. He said “Ofqual was asked by the Secretary of State to develop a system for awarding calculated grades, which maintained standards and ensured that grades were awarded broadly in line with previous years.” He is totally correct; they followed this political imperative to the letter. The fact that it was impossible to do this fairly or accurately seems to have eluded them. Why the Chief Regulator, Sally Collier, who is in charge of the Ofqual operation, was not seen is mysterious. Perhaps she has another view of the mess. 

Taylor goes on to try to reassure us with “We are already working with the Department for Education, universities and everyone else affected by this issue”. It raises the possibility that up to now they were not working with others. Why was this? 

But serious questions remain. The role of the Board of Ofqual and its governance must be brought into question. There is a large board of highly experienced individuals who must have had doubts at a very early stage. The Ofqual Board has a key role in governance and at a time of crisis it had a duty to look closely at the process. They have a very able ‘Standards Advisory Group’ that must have been informed of the progress in testing the various models that turned out to be inappropriate. It might explain why Ofqual were so worried about releasing information early about how they would proceed. 

What were the Ofqual Board doing? 

It is puzzling to find that there have been no minutes released for the Ofqual Board proceedings since 25th September of 2019 along with the minutes from 17th July of the same year. They would appear to have been very proactive in the past and meet at least seven times per year. There should have been at least seven meetings held since last September, including three in the crucial period of March, May and July of this year. These gaps in transparency and accountability must be filled as soon as possible so we can see the extent of their role in this debacle. Communications with the DfE, Williamson and the Schools Minister, Nick Gibb, should be seen. 

How bad was it and would STEM subjects be more impacted? 

Three reports from well informed experts emerged yesterday and today on the Higher Education Policy Institute www site. The first yesterday was also the last call for action made by former UCAS head, Mary Curnock-Cook who asked ‘What next for A levels and GCSEs?’. No sooner had she asked this, when the answer came back with a major U-turn. The two others amounted to ‘autopsies’ and provide well-constructed insights of the system that explain why it was doomed from the start. These concerns were raised very early on and Ofqual and the DfE should have been aware of their own failings 

Dennis Sherwood wrote positively of the U-turn with ‘CAGs rule OK’ and explained that use of teacher’s grades meant that “This year’s grades are the fairest ever.” It vindicates his position on 21st March 2020 that ‘Trusting teachers is the best way to deliver this year’s exam results – and those in future years?’ Just think how much time and angst could have been saved if he had been advising Ofqual. He also provides a concise overview of the logic behind process and its many faults with “The model was doomed from the outset”. 

This conclusion is backed up by Huy Duong who wrote today of his observations on accuracy of the exam predictions with ‘How bad were Ofqual’s grades’. The core of his evidence, after considerable detailed analysis, is in Figure 7.25 from Ofqual’s ‘Interim Report’ released last week, ’Awarding GCSE, AS, A level, advanced extension awards and extended project qualifications in summer 2020: interim report’.

It is reproduced here from the original report and I have added in pink the impact of low predicted accuracy of the ‘least bad’ model on the STEM subjects that were marked with high accuracy in 2018.

This alone should have stopped the whole process in its tracks. Later in the report they use contorted logic to claim that they could have predicted the 2019 results to ‘within one grade’ with over 90% precision. It is not acceptable as it amounts to saying that they will

 ‘get the grades wrong most of the time, but do this with great precision’. 

The crucial role of political decisions and direction. 

The Telegraph yesterday (17th August 2020 ‘Gavin Williamson and Ofqual divided over A-level exam grades’) was quick to spot some conflict between the political direction and implementation by the executive. It is hard to imagine that this was not evident much earlier in the exams process. After Williamson repeatedly defended the Ofqual approach, it appears that he was not aware of what they were actually doing in selecting the 'least bad option'. It is important that he doesn’t interfere in the workings of examinations by what should be an independent body. But by writing to them in on 31st March 20020 ‘Direction under s 129(6) of the apprenticeships, skills, children and learning act 2009’, he effectively set the agenda with ‘standardisation’ as the most important goal. He did this under his authority to direct through an Act of Parliament from 2014. 

The responsibilities of Ofqual and the Secretary of State. 

The 2014 Act is worth looking at again as it sets out the role of the Government in the process. Section 129 (6) states clearly that,

‘In performing its functions Ofqual must also have regard to such aspects of government policy as the Secretary of State may direct’. 

It seems Ofqual did this without question and, as the act also says in section (8), 

‘Ofqual must perform its functions efficiently and effectively.’ 

The fact that this was not effective was down to the fact that the government policy was flawed at the outset. Maybe another section should be added 

‘Ofqual must inform the Department of Education in a timely manner of any impediments to performing the directions of the Secretary of State'. 

There is more to consider. Under (2) it states 

‘So far as relevant, in performing its functions Ofqual must have regard to—(b) the other reasonable requirements of relevant learners, including persons with special educational needs; (c) the reasonable requirements of pupils and children, including persons with special educational needs, in relation to regulated assessment arrangements’. 

Also, under (4) Williamson had the authority to call in 

‘(a ) Her Majesty's Chief Inspector of Education, Children's Services and Skills and (b) such other relevant persons, or relevant persons of such a description, as the Secretary of State may direct’. 

The latter might have included experts nominated by the Royal Statistical Society who have been highly critics all along in a stern letter that followed considerable concerns they raised in their ‘Statement on grade adjustment in UK exams in 2020’. 

On the Ofqual side, it is required to consider,

 '(e)  the reasonable requirements of institutions within the higher education sector’.

It is clear that that Ofqual had a duty to work with the universities and colleges on the A-Levels and BTec arrangements. The evidence for this and its conclusions should be made public in their defence. What was the precise role of Universities UK in the proceedings?

A key turning point should have been evident with the findings of Robert Halfon and the Commons Education Committee who demanded that Ofqual “must urgently publish the evidence thresholds for proving bias or discrimination…. This must be communicated to parents and pupils in advance of results day” on the 11th July in their report, ‘Getting the grades they’ve earned: Covid-19: the cancellation of exams and ‘calculated’ grades’. 

It would help if the 2014 Act, that Williamson was working under, more explicitly included consideration of equality and disadvantaged groups under ‘reasonable requirements’ and thus had a duty to direct Ofqual to ensure equality. It seems this, like fairness,  was not directed and Ofqual were out on their own on this as the situation unravelled. 

Asleep at the wheel and the Ofqual ‘car crash’. 

Unless he was ‘asleep at the wheel’, all of this means that Williamson must have been aware that there would be ‘collateral damage’ arising from such an inaccurate weapon to predict grades with an algorithm. There were those who could have helped, had they been deployed effectively, and he had the authority to do this without interfering in the outcome of examinations. Instead he pressed on with the policy knowing that there would be an adverse outcome for many students. 

Today the first calls for the end of Ofqual emerged from Robert Halfon in the Guardian reported as the ‘Senior Tory MP calls for abolition of Ofqual over exam grades crisis’. Williamson seems to have survived for now and he even may last until after the GCSE results and schools, colleges and universities return to their missions. However, there is a downside to all of this. Halfon wants to see two sensible things with “It is worth considering abolishing Ofqual in its current form, it is not fit for purpose, and bringing it back within the Department for Education. Ministers should have proper control and accountability”. But his considered judgement has been ignored before, such as calls in 2018 for a Social Justice Commission, that had all party support, to replace the ill-fated Social Mobility Commission (see TEFS 24th May 2018 ‘Justice for the Social Mobility Commission: A fresh start?’. Instead, we have a government that is hell-bent on privatisation and use of the private sector. Who would take bets on BTec exam Pearson evading criticism at this time and later winning a contract from the DfE to take over the role of Ofqual. 

Then the question for classicist Boris Johnson is, who pays the ferryman? Could it be Cummings? Or is he the ferryman?

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.