Skip to main content

FOUR: Are the teachers or the students being assessed?

It seems to be generally accepted that there are major differences between schools and colleges in the success of their students in examinations. One cause could be some schools and colleges have students with better support and advantages at home. This makes such differentials inevitable. Bearing in mind better off families are willing to shell out considerable sums of money to use independent schools, it is likely that teaching and resources combine to lead to success. All things being equal otherwise, it looks like the school and teaching itself is under the microscope when examinations come along.  Hence the obsession with 'league tables'. Thus, the teacher’s ability is being assessed. The actual ability of the students seems relegated to a ghost-like’ presence hovering in the wings. 

To accompany the conclusion of no increased equality bias with examinations, Ofqual added a report on its underlying mission on standards (Monday 23rd November 2020 ‘Maintaining Standards’). This is an interesting treatise by Paul Newton, from Ofqual’s Strategy, Risk, and Research Directorate (see Note* below), who considered the logic underpinning maintenance of ‘standards’ during “normal times”, and by way of a warning about what happens “when qualifications are reformed”. There are disturbing aspects of the article that hint at the main problem of inequality in the system. 

The Sawtooth effect. 

Firstly, Newton examines in hypothetical terms the so called ‘sawtooth’ effect before and after there are reforms to examination processes. It simply shows that the performance of students in exams tends to rise over time as teachers become more familiar with the examinations and its tendencies. However, this drops immediately after the examinations are reformed. Then, performance rises again over time as teachers adapt to the new regime. This leads to shifting of grade boundaries relative to the actual performance in the old and new examinations, with boundaries drifting up over time. 

Secondly, the treatise is an unintended admission that teaching quality has a bigger influence on the outcome of examinations over time. This lies at the root of inequality whereby schools with smaller classes, more and better resources and students with family support react faster to the changes. Those at a disadvantage are slow to catch up and it explains why many parents are so wedded to using expensive, well-resourced independent schools. With exam topics released to schools in advance in England in 2021, it is not difficult to imagine those schools ramping up their provision and tutoring fast. That is what the parents pay for. There could be a significant adverse impact on attainment gap next year. 

Instead of this, on fairness Newton poses the question “If the performance of the first cohort post-reform is lower than the performance of the last cohort pre-reform, then should the new cohort of students end up with lower grades”. That some better resourced schools react faster to change is not considered. The answer is that it would not seem fair if standards are to be maintained year on year. Despite differences in ‘Attainment-Referencing’ the exams boards have resorted to making adjustments on the ‘Comparable Outcomes’ principle. However, Newton concedes “There are many outstanding challenges to address – both practical and theoretical – and it is possible that some of these challenges may never be solved.” 

Authentic or inauthentic gains. 

It could be argued that the students entering newly reformed examinations as the first cohort are largely all in the same boat. They might all be equally ill prepared. However, over time their teachers learn to adapt through what Newton calls “realignment”. This is accompanied by “adeptness” or “gains in cohort performance, over the first few years of reformed assessment arrangements, which are attributable solely to teachers becoming better at preparing learners for the new assessment structure/formats.” 

Newton makes a distinction between “authentic” gains in performance based upon content and “inauthentic” gains related to the “process by which it is assessed”. As a university teacher, I preferred students who were better informed and prepared for a performance driven assessment system. For students, this is also better rather than being adept at gaming an examination system that is radically different in a university. 

The problem I often found was that performance is affected by lack of resources and time to study, particularly those working part-time and/or commuting (TEFS 23rd August 2019 ‘Students working in term-time: Commuter students and their working patterns’

The analysis by Newton is built upon the foundation of an examination system that might change its curriculum or change the style of questioning in an examination at various times. But in doing so, he also lays out a major reason why the whole system is inherently unfair to less advantaged groups. They may be in schools that are slow to adjust. Moving the goal posts favours those ready for such a change. It seems teachers and schools are being assessed as much as the students themselves. And they may, in turn have more or fewer resources to back them.

Note*: Paul Newton is no stranger to assessments. Currently the ‘Research Chair’ at Ofqual he was professor of educational assessment at the Institute of Education, University of London. His previous positions since 1994 have been Director Cambridge Assessment Network division, Cambridge Assessment; Ofqual Head of Assessment Research; Senior Research Officer, National Foundation for Educational Research; Research Officer, Pre-school Learning Alliance; Research Associate, King's College London; Research Officer AQA (then Associated Examining Board).

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.


Popular posts from this blog

Bristol University student death: Inquest raises many concerns

The inquest into the tragic death of Bristol University Student, Ben Murray, took place this week; almost 12 months since he took his own life.* The coroner recorded a verdict of suicide earlier today but warned the University that it should make detailed inquiries after each death (BBC News ‘University of Bristol told to learn lessons after Ben Murray's suicide’ ). The anniversary of his death is this Sunday the 5th of May. Spring comes as a time of hope for most people but for others it can be a time of considerable anxiety and stress. This is especially the case for students approaching the examination period. As a close colleague of mine often pointed out, “they are all someone’s child”. Our hearts go out to the family of Ben Murray and friends as the inquest goes over again the events of a year ago. The pain is further exacerbated by media reports that he had little or no support in what was his first year at university. The BBC reported that ‘Bristol University studen

A radical overhaul of examinations is needed as soon as possible: UPDATE

UPDATE 23rd March 2021 Since this idea was posted in January, there has been considerable thought across the sector about what would be best for the future. These are very well laid out in a collection of short essays reported last week by the Higher Education Policy Institute (HEPI). The twelve essays, from different authors and different perspectives, in  ‘Where next for university admissions? ’ are edited by Rachel Hewitt  who sets out the many pitfalls surrounding examinations and university admissions. It seems there are those in favour of post qualification admission (PQA) to university as it should help the least advantaged students. However, arguments against this are presented that means caution must be taken. A powerful response to the HEPI report by the  'The Fair Access Coalition: 10 requirements for a fair admissions process' adds further to the debate. The suggestions are sensible but falls short on demanding adequate resources for students throughout their studi

Higher Education and the ‘intelligent plumbers’ theory

A common tactic when found out is to divert attention elsewhere. The release of student data from 2018/19 by the Department for Education (DfE) yesterday, ‘Widening participation in higher education: 2020’ and ‘Statistics: further education and skills’ tells the same sorry tale of a wide gap in access to universities between the most and least advantaged students. To divert attention from these stark facts in advance, the government used a diversionary tactic by attacking the effectiveness of universities and thus pointing the blame for poor social mobility someplace else. Advocating improvements in further education, something cut back by the same regime for years, hides the real intention. It seems that class divisions will be further exacerbated and any concession to universities fuelling improved social mobility has been abandoned. But the flawed theory is that at least the elite rulers will get access to intelligent plumbers . Three years ago, I heard a leading ‘You