The Ofqual University Challenge: Nobody expects the Spanish Inquisition

The evidence session of parliament's Education Committee with Ofqual this week was lengthy and depressing. Blame was put on government policy, but little seems to have changed. It must be remembered that the whole debacle happened in the context of ongoing reviews of university admission processes by Universities UK and the Office for students. With Scotland moving quickly to an inquiry, the same must happen elsewhere in the UK and it would help if this were joined up. It must move well beyond the Ofqual approach of continuing to undermine the process whilst trying to prop up a system that was already unfair and inaccurate. Moving university selection more toward potential and ability as a measure, instead of relying on attainment, should be the starting point. Then making sure every student has an equal chance must become the bedrock of a new structure. 

Senior officials from Ofqual were brought before the parliamentary Education Committee on Wednesday morning (see NOTE* below) to account for the utter confusion and chaos that surrounded the public examinations in England that also affected Wales and Northern Ireland directly. Scotland has a different system, but the same problems applied, and they are moving fast with an inquiry. 

The proceedings of nearly 2 hours and 40 minutes are available on Parliamentary TV at Education Committee, Wednesday 2 September 2020 . It was the type of detailed examination questioning over two and a half hours that every PhD graduate would understand! If it were a PhD viva, it would have got the verdict of more data needed and a major revision regarding the conclusions. They should hold off on the celebrations for a while. The full transcript is here. 

It’s not me guv (gov). 

To coincide with the hearing, the Chair of the Ofqual Board, Roger Taylor preempted the session by releasing a statement in advance ‘Written statement from Chair of Ofqual to the Education Select Committee’. It set the scene and effectively controlled the initial line of questioning by committee chair, Robert Halfon. This was an unfortunate sleight of hand but Halfon responded by echoing TEFS earlier assessment in ‘Exams 2020 and the demise of Ofqual, who pays the ferryman?’ (TEFS 18th August 2020) by summarising Ofqual's defence of their performance as it's "not me guv". He resisted use of the ironic spelling in the TEFS contribution “it wasn’t me gov”. 

From all the proceedings came an overwhelming sense that those administering the system were digging furiously without thinking much about the basis of their actions. The futility of their instructions from government should have been challenged at the outset. Instead, they were simultaneously undermining the system while trying to prop up the same unfair system and made it much worse in the process. The problem lies in defining what the assessments are for. They seem to do little else but measure educational attainment achieved through prolonged study that can be unevenly enhanced by pumping in more resources. Dare I say, like an athlete gaining advantage through use of expensive steroids and giving a false impression of ability. Surely it should be assessment on a level surface and intended to measure ability and potential. I believe everyone has now had their eyes opened about the underlying purpose and things must change. 

Do we need to know that? Mixing up education and attainment testing. 

One thing that has puzzled me for some time is the same thing that confounds efforts to educate young people arriving in a university. They mostly respond in a logical way to any challenge and do what is required to succeed. The regime in schools, that appears entirely focused on passing exams to succeed, does not transfer well into a university setting. New students very often asked me a simple question, "do we need to know that?" On the one hand it is a deep philosophical question pertaining to knowledge in general. Answering on that level leads to some puzzlement from the student. It is difficult to say if anything we learn needs to be learned until it has some use. We may never need to deploy the knowledge, or we may find it of use in the next day or so, who can say with certainty? I give examples in the science context such as, did you know that 50 drops from a ‘sawn-off’ 18-gauge syringe needle comes to exactly 1 ml or 1,000 µl, usually within one drop and 2% accuracy? It is a way to measure small volumes from 20 µl upwards with incredible accuracy. It is a very old method and it can be sterilised and reused by dipping in alcohol and flaming it off. But do you need to know that? Well, if in an isolated area, with few supplies and no access to modern laboratory equipment, then using a common syringe needle to carry out accurate biochemical tests would be very useful. In a modern laboratory you would use a calibrated pipettor with standard disposable tips. It they are not properly calibrated, then the dropper method is more reliable. This offers an insight into simple thinking about measurements and how to adapt to situations when a need arises. It sits at the core of science education in a university. 

Of course, what the student means is “do we need to know that for the exam?” They are so conditioned to preparing for examinations that the idea of independent thinking seems to have been drained from many of them. When I ask, has anyone at school ever told you "you don’t need to know that", they all say "yes, this is normal". 

This illustrates the fundamental conflict of working to the examination versus offering a way of thinking through a problem, and thus providing ‘education’. In this way, a university education becomes a great leveller. Analytical thought, creativity and problem solving become the dominant educational goals. It is why many universities conclude that A-level or Higher grades are not necessarily a good predictor of success in a university. Those grades may have been achieved under very different contexts and circumstances. An A grade student in a heavily resourced school simply has it easier, with access to information and help readily at hand. An A grade student from a crowded comprehensive school, and little help at home, would have shown heroic ingenuity, stamina, and endeavour. Either student could turn out to be a ‘bloody genius’ at university, but one is more likely to show this if given the same amount of time to study and the same opportunity. The problem is that success is still linked to the amount of resource and time put into studies and assignments. Those with less time due to commuting or part-time jobs are still heavily disadvantaged (see TEFS 16th June 2020 ‘University student part-time working is a dangerous blind spot’) 

Separating testing from education. 

Conflating education with testing too often leads to confusion about what should dominate, a sound education or the test. Alarmingly, the idea of the 'test' being the same as ‘education’ dominates, even in the minds of those at Ofqual. 

There is no doubt that  achieving attainment dominates our current system. This is done by loading up the students with as much information as possible, so they are prepared for a test at the end. The concern about students falling behind is rooted in having more time to fit in more. It leads to the inevitable outcome that those with more resources get more help at this task and are better prepared. Those with fewer resources in larger classes are at a disadvantage. 

As I was writing, and got to this this part yesterday, I noticed on Twitter that Steven Jones, Head of the Manchester Institute of Education, had just released an article, ‘Might selective English universities now wean themselves off A-levels?’ on the Council for the Defence of British Universities (CDBU) www site. It strongly reinforces my observations with the idea of universities being ‘addicted‘ to the exclusivity of narrow A-level attainment. He essentially took the words right out of my mouth with, 

“Like most seasoned university lecturers, I have taught students with stellar grades who struggle with the idea of learning independently and thinking critically. I have also taught students without A-levels who blossom on campus, swiftly figuring out how to absorb complex, contrary positions and craft measured, evidence-based arguments”. 

If any lecturer says that they have not observed this, then they cannot be paying much attention. Ability is something separate from attainment. Thus, it is better if universities focused on ability first to admit students to university courses. 

Looking at alternatives. 

There are alternatives, but it is not likely that the established government would ever grasp that nettle. In ‘Working through college: A tale of two cities’, TEFS (3rd July 2020) made some observations about the support structures in the USA and funding linked to basic ability and attainment. 

The relative value of alternatives was also discussed by TEFS (19th April 2019) in ‘Grade inflation and contextualised admissions to university are stirring up a wasp’s nest’. It is worth revisiting the arguments in the context of the A-level mess that emerged this summer. 

The USA relies on a combination of different measures to assess candidates for university or college entry. Measures such as the high school ‘Grade Point Average’ (there is a good explanation from the Higher Education Academy of how it works generally in ‘Guide on grade point average (GPA) for students’) are important but there are considerable concerns about standardisation across different jurisdictions and fairness (see ‘Grade point average: what's wrong and what's the alternative?’ Journal of Higher Education Policy and Management Volume 33, 2010, 27-36). 

This was realised nearly 100 years ago and a standard test, called the ‘Scholastic Aptitude Test’ or SAT was devised in 1926. It has seen many changes and has evolved over the years. The idea was to look at aptitude on top of attainment and act as a leveller. I completed such a test in London in 1977 and found it did not require any special tutoring. Looking now it is clear that changes have been made and further radical reforms were made in 2016. It seems to have evolved into a different species. The original idea of testing outside of the school curriculum was watered down considerably. You can try some example questions at the USA College Board www site. I wonder if I could have done as well in the reformed test (?). Currently, the final score is determined by a combination of reading, writing and a maths test with an optional essay. It is designed to eliminate rote learning as much as possible, but tutoring would enhance the score. It is clear that a very sound use of English is necessary, even to understand some of the maths questions. This means that it remains inherently socially biased towards those with a sophisticated and nuanced understanding of English that they could acquire through tutoring or a better learning environment at home. 

The inevitable outcome is that universities across the USA are considering abandoning the new SAT and trying other means to judge ability. In May, the university of California decided to drop SAT (and its close cousin, ACT or American College Testing) from their entrance criteria (see ‘University of California to drop SAT and ACT as admission requirements’ The Guardian 22nd May 2020). The conclusion was that they were seeking to avoid discriminating against disadvantaged students. In the UK, we might say that the tests had ‘mutated’. 

What is happening in the UK? 

In 2004, the Higher Education Steering Group in the UK also looked at alternatives in ‘Fair admissions to higher education: recommendations for good practice’. This included consideration of additional tests such as the USA based SAT that “may help to uncover hidden talent”. Indeed the Sutton Trust carried out a rigorous comparison between the SAT and A-Levels in 2001 in ‘A Pilot of Aptitude Testing for University Entrance’ with tantalising conclusions. Although there was a general correlation evident in the comparison of A-Level grades and SAT score, the extent of ‘scatter’ across those from low-achieving, high-achieving and independent schools was an alarming feature of the data. A significant number of students in low-achieving schools had high SAT scores and yet achieved very low A-Level scores. That some of these might then thrive at university, when given some fair resources and a chance, should surely come as no surprise. 

It took until 2019 for Universities UK (UUK) to act with a ‘Major review of university admissions’. But they asked if the 2004 “principles on fair admissions in higher education remain valid”. This means that they were questioning the validity of the stated principles of “• Transparency; • Selection for merit, potential and diversity; • Reliability, validity and relevance; • The minimising of barriers; • Professionalism”. That was not a good starting point. 

Meanwhile, in response, the Office for Students (OfS) began its own ‘Reviewing the admissions system’ with similar principles that was launched at the end of February of this year. It is due to report soon this Autumn. However, this would only apply to England. They must surely conclude that the intervening months have tested the current system to destruction. Seeing as the current examination debacle was acted out in the context of these reviews, it means that lessons have still to be learned. An urgent new review is needed that spans all the UK 

The links to the rest of the UK are now a major issue. 

With the UUK and OfS reviewing admissions, it seems there are likely to be conflicts that resonate across the other UK jurisdictions. The Education Committee hearing avoided asking awkward questions about the examination decisions made in Northern Ireland and Wales and what communications had taken place. The inference is that both jurisdictions went along with Ofqual for their examinations – despite reservations. Those reservations should also be made public as part of a review. However, the reason for maintaining uniformity is a simple one. Many schools in Northern Ireland and Wales opt to enter examinations set by exam boards in England under the regulatory control of Ofqual. They had no choice but to string along with the plan. Scotland is very different. Their public examinations of National 5, Higher and Advanced Higher are radically different. This may account for the differences in approach made by SQA in Scotland this summer. However, the principle of maintaining standards was also adopted as a political decision early on. As soon as the results started to emerge, the Scottish Government took decisive action and abandoned ship. The writing was on the wall for Ofqual and the others across the UK and the edifice came tumbling down. 

Practical solutions.

There seem to be two logical approaches at either end of the action spectrum that should be considered. Taking no action is certainly no longer an option. 

At one end, we could work with the system that exists and seek to find more support for disadvantaged students to reach attainment levels that their better resourced peers find easier to reach. At the other end, we might simply reform the assessment system to look for potential and ability as the differential between students. The latter is likely to be resisted and controversial. Fortunately, there are some brave souls who seek to level the field for students with actions that require more help and resources. However, the odds will be stacked against them as those with more seek to move goalposts. Former Education Secretary, Justine Greening, is one such person with the determination to make change. In iNews on Wednesday, she offered some salient advice with ‘Ministers love to launch new projects – but my 2017 education policy can still save the class of 2020’. She reminded us of her approach back in 2017 when she was the Education Secretary. With “Now it’s time to move beyond the problems to finding solutions” she pleads for an enhancement of her initiative ‘Opportunity area programme’. It covered a limited number of twelve areas and offered hope to people in those areas only. Greening is looking to extend the concept wider afield and at least go some way to bridging the opportunity divide. It looks like it will succeed on that level. 

Ruth Dacey with The Yorkshire Post in ‘Widening Education Gap’ observed that the initiative was closing the attainment gap slowly in those areas fortunate to get the extra help. It tends to confirm the conclusion that, if enough resources are diverted into educational support, then the attainment rises. This is something that families buying private education have known for a long time. But most observers think that the government’s objective is to keep the differential in place to support the premium offered by independent schools. Greening was highly effective in government; but note she did not last long and was replaced in 2018 and is no longer in government (see TEFS 25th January 2019 ‘Entering the Lion’s Den: Can the Conservative establishment be tamed?’). Her approach is only one side of the effort that is needed to fully succeed. 

The solution probably lies someplace in between. Universities need to know that their students have some knowledge of the subject before they arrive. However, this must not be the only criterion. Their success cannot be sustained by simply adding more weight onto their intellectual structure. There needs to be a more solid foundation of ability and potential at the outset, and determination and interest thrown it. Ability and potential are the most important qualities needed in a University. Knowledge will become dated as advances are made; this is particularly the case in science. Having the clear potential to cope with change and, better still, effect change should be the decider.

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.

NOTE* Education Committee evidence hearing Wednesday 2nd September 2020. The impact of COVID-19 on education and children’s services. Witness(es): Roger Taylor, Chair, Ofqual; Dr Michelle Meadows, Executive Director for Strategy and Research, Ofqual; Julie Swan, Executive Director, General Qualifications; Dame Glenys Stacey, Acting Chief Regulator, Ofqual