Skip to main content

FOUR: Are the teachers or the students being assessed?

It seems to be generally accepted that there are major differences between schools and colleges in the success of their students in examinations. One cause could be some schools and colleges have students with better support and advantages at home. This makes such differentials inevitable. Bearing in mind better off families are willing to shell out considerable sums of money to use independent schools, it is likely that teaching and resources combine to lead to success. All things being equal otherwise, it looks like the school and teaching itself is under the microscope when examinations come along.  Hence the obsession with 'league tables'. Thus, the teacher’s ability is being assessed. The actual ability of the students seems relegated to a ghost-like’ presence hovering in the wings. 

To accompany the conclusion of no increased equality bias with examinations, Ofqual added a report on its underlying mission on standards (Monday 23rd November 2020 ‘Maintaining Standards’). This is an interesting treatise by Paul Newton, from Ofqual’s Strategy, Risk, and Research Directorate (see Note* below), who considered the logic underpinning maintenance of ‘standards’ during “normal times”, and by way of a warning about what happens “when qualifications are reformed”. There are disturbing aspects of the article that hint at the main problem of inequality in the system. 

The Sawtooth effect. 

Firstly, Newton examines in hypothetical terms the so called ‘sawtooth’ effect before and after there are reforms to examination processes. It simply shows that the performance of students in exams tends to rise over time as teachers become more familiar with the examinations and its tendencies. However, this drops immediately after the examinations are reformed. Then, performance rises again over time as teachers adapt to the new regime. This leads to shifting of grade boundaries relative to the actual performance in the old and new examinations, with boundaries drifting up over time. 

Secondly, the treatise is an unintended admission that teaching quality has a bigger influence on the outcome of examinations over time. This lies at the root of inequality whereby schools with smaller classes, more and better resources and students with family support react faster to the changes. Those at a disadvantage are slow to catch up and it explains why many parents are so wedded to using expensive, well-resourced independent schools. With exam topics released to schools in advance in England in 2021, it is not difficult to imagine those schools ramping up their provision and tutoring fast. That is what the parents pay for. There could be a significant adverse impact on attainment gap next year. 

Instead of this, on fairness Newton poses the question “If the performance of the first cohort post-reform is lower than the performance of the last cohort pre-reform, then should the new cohort of students end up with lower grades”. That some better resourced schools react faster to change is not considered. The answer is that it would not seem fair if standards are to be maintained year on year. Despite differences in ‘Attainment-Referencing’ the exams boards have resorted to making adjustments on the ‘Comparable Outcomes’ principle. However, Newton concedes “There are many outstanding challenges to address – both practical and theoretical – and it is possible that some of these challenges may never be solved.” 

Authentic or inauthentic gains. 

It could be argued that the students entering newly reformed examinations as the first cohort are largely all in the same boat. They might all be equally ill prepared. However, over time their teachers learn to adapt through what Newton calls “realignment”. This is accompanied by “adeptness” or “gains in cohort performance, over the first few years of reformed assessment arrangements, which are attributable solely to teachers becoming better at preparing learners for the new assessment structure/formats.” 

Newton makes a distinction between “authentic” gains in performance based upon content and “inauthentic” gains related to the “process by which it is assessed”. As a university teacher, I preferred students who were better informed and prepared for a performance driven assessment system. For students, this is also better rather than being adept at gaming an examination system that is radically different in a university. 

The problem I often found was that performance is affected by lack of resources and time to study, particularly those working part-time and/or commuting (TEFS 23rd August 2019 ‘Students working in term-time: Commuter students and their working patterns’

The analysis by Newton is built upon the foundation of an examination system that might change its curriculum or change the style of questioning in an examination at various times. But in doing so, he also lays out a major reason why the whole system is inherently unfair to less advantaged groups. They may be in schools that are slow to adjust. Moving the goal posts favours those ready for such a change. It seems teachers and schools are being assessed as much as the students themselves. And they may, in turn have more or fewer resources to back them.

Note*: Paul Newton is no stranger to assessments. Currently the ‘Research Chair’ at Ofqual he was professor of educational assessment at the Institute of Education, University of London. His previous positions since 1994 have been Director Cambridge Assessment Network division, Cambridge Assessment; Ofqual Head of Assessment Research; Senior Research Officer, National Foundation for Educational Research; Research Officer, Pre-school Learning Alliance; Research Associate, King's College London; Research Officer AQA (then Associated Examining Board).

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.


Popular posts from this blog

A radical overhaul of examinations is needed as soon as possible: UPDATE

UPDATE 23rd March 2021 Since this idea was posted in January, there has been considerable thought across the sector about what would be best for the future. These are very well laid out in a collection of short essays reported last week by the Higher Education Policy Institute (HEPI). The twelve essays, from different authors and different perspectives, in  ‘Where next for university admissions? ’ are edited by Rachel Hewitt  who sets out the many pitfalls surrounding examinations and university admissions. It seems there are those in favour of post qualification admission (PQA) to university as it should help the least advantaged students. However, arguments against this are presented that means caution must be taken. A powerful response to the HEPI report by the  'The Fair Access Coalition: 10 requirements for a fair admissions process' adds further to the debate. The suggestions are sensible but falls short on demanding adequate resources for students throughout their studi

Qfqual builds a concrete wall: UPDATED

UPDATE 8th August 2020 Things are moving fast today with severe criticism mounting about Ofqual and SQA, and urgent action is needed. TEFS has laid out ten points that should be considered to reverse out of the crumbling mess. Fairness should replace 'maintaining standards' as the primary objective. The government must cease trying to defend a system that acts as a barrier to the less advantaged. Since posting yesterday, things have been moving fast. Today the Guardian put the examinations issue in large print on its front page with ‘Nearly 40% of A-level result predictions to be downgraded in England’ . This conclusion came about after some great detective work by former medical statistician, Huy Duong, who analysed the data available and reconciled this with the Ofqual announcement that there could have been a 12% inflation in higher grades. It seems that Ofqual have been caught red handed and "Duong’s findings were privately confirmed to the Guardian by ex

The next labour of Ofqual is announced: Social mobility UPDATE

UPDATE 1st March2021  Since writing this post, there has been valuable analysis added to the worsening situation by Lee Elliot-Major, Chair of Social Mobility at Exeter University and former head of the Sutton Trust. His article in The Guardian today, ‘How do we ensure disadvantaged kids don't lose out in England's new exam system?’  concludes that “it will be long after this summer’s exam grade battles that we will comprehend the full consequences this pandemic has had on young people.” That could be an understatement as the idea of ‘social mobility’ unravels fast. He cites a recent research publication with colleagues at the LSE Centre for Economic Performance  entitled  ‘Unequal learning and labour market losses in the crisis: consequences for social mobility’ . This is a detailed and rigorous analysis and survey that should set alarm bells ringing in government in the run-up to the budget this week. The evidence is stark as the “education and labour market losses due to C