Ofqual lets the cat out of the bag

Ofqual concluded this week that they had little option but to release the minutes of their board meetings. The news broke yesterday, and they are now in the public domain (Ofqual board minutes for 2020 and Ofqual board minutes for 2019). They make for sobering and very depressing reading as the extent of confusion and conflict between Ofqual and the government is revealed.

As time progressed, and students were sent their results, the public became aware of the extent of the blunder. The Scottish government was the first to realise the enormity of the error they had made and took full responsibility at the political level. The administrations in England, Wales and Northern Ireland capitulated in their own way. There was utter confusion in England, poor communication, and a blame game. It seems the minutes released an angry 'cat out of the bag'. The actions of the government in England probably reflect their reluctance to let go of their ‘standardisation’ process that maintains the status quo and existing social divisions. They were fully aware of the dangers but were prepared to ‘sacrifice’ the progress and careers of many disadvantaged students to achieve this goal. COVID has opened our eyes to the tensions and revealed the perverse aims of the government. 

TEFS reported multiple Ofqual Board meetings last week.

Last week, TEFS reported that an earlier freedom of information request revealed 32 board meetings since September of last year (TEFS 16th October 2020 ‘Ofqual holding back information’). The response of TEFS was considered by Ofqual Officials and they were very concerned about their position. The extent of Ofqual Board involvement in the chaos of the examinations process was already illustrated by Ofqual declaring 29 meetings since March of this year. A staggering 26 of these were emergency meetings that included three on Sunday 16th August, the day before the government made a radical U-turn on the grades already sent to students. The stark conclusion from the minutes is that Ofqual was prepared to press on with its ‘standardisation’ and downgrading of students’ grades, with appeals being the only fall-back position. Then it further thought falling back on mock results would inspire confidence. This was simply astounding as the Secretary of State continually interfered. The lines of responsibility were blurred as Ofqual was put in an impossible situation. 

The record is still incomplete. 

The minutes are not shown in full since they are littered with a significant number of redactions as TEFS had predicted. These are indicated with statements such as: “This section has been redacted on the basis that it contains legally privileged information” or “This section has been redacted, as its publication would be prejudicial to the effective conduct of public affairs.” We will never know what remains hidden, but there is a suspicion that many refer to communications with the Secretary of State and the Department of Education. 

Conflict and the root cause. 

When reading the minutes yesterday, there was an overwhelming sense that the chaos started with a misguided directive from the government. I say ‘government’ since this decision cannot have been made by the Secretary of State for Education alone. Instead, such a key decision must have been agreed with, or even directed by, No10. This set a pattern of conflict and chaos in motion. The minutes show that Ofqual went to extraordinary lengths to meet the directive from the government. 

Right from the outset, it became clear the Secretary of State for Education, Gavin Williamson, was in the policy driving seat. His letter of 31st of March 2020 to the soon to resign Chief Regulator, Sally Collier, stressed that maintaining standards was the top propriety with “it is Government policy that these students should be issued with calculated results based on their exam centres”. Despite his later assertions at an evidence hearing of the Education Committee that he was concerned about fairness and access for disadvantaged students, his instructions to Ofqual in March did not mention this. Instead, he stressed to the committee that: 

“If you go back to my correspondence and direction letter on the 31st March it did rightly highlight the importance of – ah, you know – sort of children from disadvantaged and ethnic minority backgrounds. It also highlighted the importance of standards as well.” 

This was not correct (See TEFS 18th September 2020 ‘Three Wise Monkeys’ of the exams mess’). 

Williamson also said he was not aware of the problems arising from his directive until the 15th of August and seems to have been imaginative in hindsight about the impact on students from less advantaged areas. The minutes indicate otherwise since his ‘fingerprints’ are all over the process. 

Pivotal meeting on 13th of May 2020. 

The minutes of the 13th of May indicated that the strategy to be adopted was to be cemented in place. Ofqual had accepted an impossible task from the government and pushed on despite opposition. This was a dreadful mistake with some details yet to be revealed. 

“The Executive Director for General Qualifications (GQs) reported that the main representative groups, [this section has been redacted, as its publication would be prejudicial to the effective conduct of public affairs] had been supportive of the proposal that centre trajectory should not be part of the standardisation model. However, there had been pushback from some of the larger Multi Academy Trust (MAT) chains."

There was also a worry that schools and colleges might not be trusted in their assessments and this drew Ofqual into the mire of a less than open or professional approach. The result was an over reliance on the algorithm for calculating individual grades using a ‘standardised model’. 

“With regard to the standardisation model, the Board resolved to agree: That the proposed aims of the standardisation model are adopted but that they are reordered such that the aim regarding the method’s transparency and simplicity, appears at the end of the list so as to not overstate the importance of simplicity over likely accuracy”  and "To consider the timing of a publication on the details of the standardisation model” 

The effect of a Scottish decision and nine emergency meetings between the 11th and 16th of August 2020. 

If the die was cast on the 13th of May, then it was thrown away in a series of meetings in August. These magnified the confusion and conflict that was brewing earlier. After Williamson and the government was seriously blindsided by the Scottish government decision to drop its assessment method, it was inevitable that the rest of the UK would have to fall in line. 

The Scottish Government recognised the problem quickly as confidence in their standardised grades had evaporated days earlier. Their decision to move to grades assessed in Schools and Colleges (Centre Assessed Grades) was a major factor in influencing the administrations in the rest of the UK. Indeed, Williamson intervened directly in the business of Ofqual by phone on 11th August and the Board was clear later the same day that “The meeting had been convened at short notice to consider options which had been raised earlier that morning by the Secretary of State for Education (SoS) to address declining public confidence in calculated grades and the events in Scotland.” It would have been easier for everyone if Williamson had just caved in as the Scottish Government had. Instead, he was determined to maintain standards and rely on appeals and mock results. Ofqual acted in the way they thought was intended and duly complied. 

They were then left drifting since “Ofqual was not in possession of a policy position from the Department at this time”. This sowed the seeds of major confusion. They thought that Williamson had “indicated a preference for a fast track way for students to receive their mock result instead of their calculated grade”. Conflict was inevitable since “The Board noted that this was a difficult position for Ofqual. The Board had previously received legal advice concerning the SoS’ power of direction and Ofqual’s duties in that regard”. 

Despite serious reservations, Ofqual had little choice but to do what they did and issued guidance. It did not last long. By the meeting of the 15th August 2020, “The Board resolved to agree to remove the statement from the website, pending further advice and consideration, and to issue a holding statement saying that the Board was reviewing the appeals process”. 

They persisted in their task and agreed that “The Board would reconvene on 16 August 2020 to consider again the options outlined in the paper presented at the Emergency Board Meeting on 14 August 2020 and to be further informed and updated on any further communications between the Chief Regulator, Chair and the SoS"  then "This section has been redacted, as its publication would be prejudicial to the effective conduct of public affairs”. 

The U-turn was almost complete by the third meeting on the evening of Sunday 16th August 2020. The decision to accede to government pressure was taken in the context of  “The current state of public opinion as reported in the media as well as the context which is provided by the decision taken for Learners in Scotland and Northern Ireland.” 

The board was already aware of her decision when Chief Regulator, Sally Collier later resigned and walked away (BBC News 25th August 2020 ‘Ofqual chief Sally Collier steps down after exams chaos’).

Acknowledgement of disadvantaged groups. 

Despite fears that disadvantaged groups would be adversely affected, the following decision also made on the 13th of May 2020. 

“The Executive Director for Strategy, Risk and Research stated that a number of principles were being developed to refine the statistical model. That included proposals that the statistical standardisation model should place more weight on historical evidence of centre performance than the submitted centre assessment grades where it would result in students being awarded the grades they would most likely had achieved had they been able to complete their assessments in summer 2020. Consideration would need to be given to monitoring the model and any potential impacts it may have, and the steps (if any) Ofqual could legitimately take to mitigate those impacts on disadvantaged groups.” 

Of course, it was already too late, and the die was cast. The government was getting the standardisation they wanted.

Being prepared and involvement of expert advisors. 

In their FOI response to TEFS, Ofqual admitted they were not prepared for the likelihood of an event such as a pandemic, or other natural disasters, closing multiple exam centres with “Ofqual had no contingencies in place for the exceptional circumstances that have affected the examination process during a potential lock down”. This is very surprising since a pandemic was predicted by scientists for years. If any consideration had been given to such a scenario, then the confusion, and unacceptable outcome from a wrong political decision, would have been apparent. Indeed, the discriminatory bias toward the most advantaged inherent in the whole examination process would have been revealed much sooner. 

What is also missing is evidence and input from the highly experienced special advisors that make up the Ofqual ‘Standards advisory group’. These are people involved in education and they would have been highly critical of the approach taken and its dangers. The Ofqual Board itself is much less experienced in education as a whole and many members should have resisted dabbling in educational matters outside of their professional sphere. 

The lack of involvement of a very experienced advisory team was also counter to the decision of the board on the 13th May 2020  “To delegate final decisions as to the operation of the standardisation model to Michelle Meadows, Executive Director for Strategy, Risk and Research, in discussion with Mike Cresswell as Chair of the External Advisory Group" (TEFS note AKA the ‘Standards Advisory Group’).

The FOI reply to TEFS, revealed last week, showed that there was only one meeting of the Standards Advisory Group in 2020, and that was only on the 22nd September 2020. The minutes have yet to be released. This meeting was long after “a briefing from Ofqual officials on Summer 2020 Results on 05 August 2020”. Their conclusions at such a late stage would be of great interest. 

Incompetence? 

Some will conclude that the confusion was born out of incompetence by the government or Ofqual or both. This is a mistake and is a convenient mask for ministers to hide behind. Instead, it is better to conclude that the decision to maintain standards was a wilful act with the intention of maintaining the status quo. Indeed, the system used for calculating grades, based on the past performances of schools and colleges, ensures that the most advantaged remain in pole position in the educational race. The minutes reveal that Williamson and the government were aware of the consequences of their action in generating a significant number of outliers and skewing the results in favour of the most advantaged. On the other hand, some in Ofqual were very concerned but the directive from Williamson was overriding such concerns. 

Why was the Ofqual Board so deeply involved in the decisions? 

The direct involvement of the board at so many points in the process is unprecedented when looking back at the records of previous meetings. One reason for this appears to be the relative ease in arranging meetings online using Microsoft Teams. This has indeed become a feature of the decision-making operations of many organisations. The online facilities allow meetings to run almost in a continuous manner, bringing in participants at will. The result is a sort of drift and a lack of focus spawned by constant revisiting earlier decisions. Couple this with a bad decision at the outset and confusion was inevitable. 

Confidence in the examination process, and those making the decisions, is now very low. The COVID crisis revealed and magnified an inherent bias towards those with more advantages. The government deployed Ofqual as a convenient vehicle to carry out a process of ‘standardisation’ in the full knowledge that it would exacerbate the situation. They almost got away with it but were apprehended by the free press and others who could ‘smell a rat’. It is time to radically reform the examination system and make it fairer. Next year’s round of examinations will be observed by the public with their eyes fully open. The system will be tested as never before.

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.