Skip to main content

FOUR: Are the teachers or the students being assessed?

It seems to be generally accepted that there are major differences between schools and colleges in the success of their students in examinations. One cause could be some schools and colleges have students with better support and advantages at home. This makes such differentials inevitable. Bearing in mind better off families are willing to shell out considerable sums of money to use independent schools, it is likely that teaching and resources combine to lead to success. All things being equal otherwise, it looks like the school and teaching itself is under the microscope when examinations come along.  Hence the obsession with 'league tables'. Thus, the teacher’s ability is being assessed. The actual ability of the students seems relegated to a ghost-like’ presence hovering in the wings. 

To accompany the conclusion of no increased equality bias with examinations, Ofqual added a report on its underlying mission on standards (Monday 23rd November 2020 ‘Maintaining Standards’). This is an interesting treatise by Paul Newton, from Ofqual’s Strategy, Risk, and Research Directorate (see Note* below), who considered the logic underpinning maintenance of ‘standards’ during “normal times”, and by way of a warning about what happens “when qualifications are reformed”. There are disturbing aspects of the article that hint at the main problem of inequality in the system. 

The Sawtooth effect. 

Firstly, Newton examines in hypothetical terms the so called ‘sawtooth’ effect before and after there are reforms to examination processes. It simply shows that the performance of students in exams tends to rise over time as teachers become more familiar with the examinations and its tendencies. However, this drops immediately after the examinations are reformed. Then, performance rises again over time as teachers adapt to the new regime. This leads to shifting of grade boundaries relative to the actual performance in the old and new examinations, with boundaries drifting up over time. 

Secondly, the treatise is an unintended admission that teaching quality has a bigger influence on the outcome of examinations over time. This lies at the root of inequality whereby schools with smaller classes, more and better resources and students with family support react faster to the changes. Those at a disadvantage are slow to catch up and it explains why many parents are so wedded to using expensive, well-resourced independent schools. With exam topics released to schools in advance in England in 2021, it is not difficult to imagine those schools ramping up their provision and tutoring fast. That is what the parents pay for. There could be a significant adverse impact on attainment gap next year. 

Instead of this, on fairness Newton poses the question “If the performance of the first cohort post-reform is lower than the performance of the last cohort pre-reform, then should the new cohort of students end up with lower grades”. That some better resourced schools react faster to change is not considered. The answer is that it would not seem fair if standards are to be maintained year on year. Despite differences in ‘Attainment-Referencing’ the exams boards have resorted to making adjustments on the ‘Comparable Outcomes’ principle. However, Newton concedes “There are many outstanding challenges to address – both practical and theoretical – and it is possible that some of these challenges may never be solved.” 

Authentic or inauthentic gains. 

It could be argued that the students entering newly reformed examinations as the first cohort are largely all in the same boat. They might all be equally ill prepared. However, over time their teachers learn to adapt through what Newton calls “realignment”. This is accompanied by “adeptness” or “gains in cohort performance, over the first few years of reformed assessment arrangements, which are attributable solely to teachers becoming better at preparing learners for the new assessment structure/formats.” 

Newton makes a distinction between “authentic” gains in performance based upon content and “inauthentic” gains related to the “process by which it is assessed”. As a university teacher, I preferred students who were better informed and prepared for a performance driven assessment system. For students, this is also better rather than being adept at gaming an examination system that is radically different in a university. 

The problem I often found was that performance is affected by lack of resources and time to study, particularly those working part-time and/or commuting (TEFS 23rd August 2019 ‘Students working in term-time: Commuter students and their working patterns’

The analysis by Newton is built upon the foundation of an examination system that might change its curriculum or change the style of questioning in an examination at various times. But in doing so, he also lays out a major reason why the whole system is inherently unfair to less advantaged groups. They may be in schools that are slow to adjust. Moving the goal posts favours those ready for such a change. It seems teachers and schools are being assessed as much as the students themselves. And they may, in turn have more or fewer resources to back them.

Note*: Paul Newton is no stranger to assessments. Currently the ‘Research Chair’ at Ofqual he was professor of educational assessment at the Institute of Education, University of London. His previous positions since 1994 have been Director Cambridge Assessment Network division, Cambridge Assessment; Ofqual Head of Assessment Research; Senior Research Officer, National Foundation for Educational Research; Research Officer, Pre-school Learning Alliance; Research Associate, King's College London; Research Officer AQA (then Associated Examining Board).

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.


Popular posts from this blog

Ofqual holding back information

Ofqual has responded to an FOI request from TEFS this week. They held a staggering twenty-nine board meetings since March. Despite promising the Parliamentary Education Committee over a month ago they would publish the minutes “shortly” after their meeting on 16th September, they are still not able to do so. They cite “exemption for information that is intended to be published in the future” for minutes that are in the “process of being approved for publication” . More concerning is they are also citing exemption under the “Public Interest Test”. This means they might not be published, and Ofqual will open themselves up to legal challenges. If both the Department for Education and Ofqual are prevented from being more open, then public interest will lie shattered on the floor and lessons will not be learned.  Ofqual finally responded to the TEFS Freedom of Information (FOI) request to publish the minutes of its board meetings on Tuesday. It should have been replied to by 17th Septembe

Research and Teaching: The price of researchers not teaching

Has the emphasis on research and REF damaged the teaching and support for students by front-line academic staff? "Your teaching is trivial, anyone can do it”.   After 35 years of academic life as a scientist teaching Microbiology and Biochemistry in a Russell Group university, this is exactly what a ‘senior’ manager told me as I negotiated my ‘smooth’ career exit when I decided to retire early. I left at the end of 2016 after spending a year working part-time whilst winding down my research group. I offered to continue with some teaching over that time simply to help out colleagues. But that was the response to my offer and my teaching ended more abruptly than expected in the summer of 2015.  How had things come to this?  The remark was probably a careless throw away comment but it was made by someone more ‘senior’ to me who had comparatively little experience of teaching undergraduate students. It betrayed an attitude that had been gaining ground for some time. I wondered