Skip to main content

FOUR: Are the teachers or the students being assessed?

It seems to be generally accepted that there are major differences between schools and colleges in the success of their students in examinations. One cause could be some schools and colleges have students with better support and advantages at home. This makes such differentials inevitable. Bearing in mind better off families are willing to shell out considerable sums of money to use independent schools, it is likely that teaching and resources combine to lead to success. All things being equal otherwise, it looks like the school and teaching itself is under the microscope when examinations come along.  Hence the obsession with 'league tables'. Thus, the teacher’s ability is being assessed. The actual ability of the students seems relegated to a ghost-like’ presence hovering in the wings. 

To accompany the conclusion of no increased equality bias with examinations, Ofqual added a report on its underlying mission on standards (Monday 23rd November 2020 ‘Maintaining Standards’). This is an interesting treatise by Paul Newton, from Ofqual’s Strategy, Risk, and Research Directorate (see Note* below), who considered the logic underpinning maintenance of ‘standards’ during “normal times”, and by way of a warning about what happens “when qualifications are reformed”. There are disturbing aspects of the article that hint at the main problem of inequality in the system. 

The Sawtooth effect. 

Firstly, Newton examines in hypothetical terms the so called ‘sawtooth’ effect before and after there are reforms to examination processes. It simply shows that the performance of students in exams tends to rise over time as teachers become more familiar with the examinations and its tendencies. However, this drops immediately after the examinations are reformed. Then, performance rises again over time as teachers adapt to the new regime. This leads to shifting of grade boundaries relative to the actual performance in the old and new examinations, with boundaries drifting up over time. 

Secondly, the treatise is an unintended admission that teaching quality has a bigger influence on the outcome of examinations over time. This lies at the root of inequality whereby schools with smaller classes, more and better resources and students with family support react faster to the changes. Those at a disadvantage are slow to catch up and it explains why many parents are so wedded to using expensive, well-resourced independent schools. With exam topics released to schools in advance in England in 2021, it is not difficult to imagine those schools ramping up their provision and tutoring fast. That is what the parents pay for. There could be a significant adverse impact on attainment gap next year. 

Instead of this, on fairness Newton poses the question “If the performance of the first cohort post-reform is lower than the performance of the last cohort pre-reform, then should the new cohort of students end up with lower grades”. That some better resourced schools react faster to change is not considered. The answer is that it would not seem fair if standards are to be maintained year on year. Despite differences in ‘Attainment-Referencing’ the exams boards have resorted to making adjustments on the ‘Comparable Outcomes’ principle. However, Newton concedes “There are many outstanding challenges to address – both practical and theoretical – and it is possible that some of these challenges may never be solved.” 

Authentic or inauthentic gains. 

It could be argued that the students entering newly reformed examinations as the first cohort are largely all in the same boat. They might all be equally ill prepared. However, over time their teachers learn to adapt through what Newton calls “realignment”. This is accompanied by “adeptness” or “gains in cohort performance, over the first few years of reformed assessment arrangements, which are attributable solely to teachers becoming better at preparing learners for the new assessment structure/formats.” 

Newton makes a distinction between “authentic” gains in performance based upon content and “inauthentic” gains related to the “process by which it is assessed”. As a university teacher, I preferred students who were better informed and prepared for a performance driven assessment system. For students, this is also better rather than being adept at gaming an examination system that is radically different in a university. 

The problem I often found was that performance is affected by lack of resources and time to study, particularly those working part-time and/or commuting (TEFS 23rd August 2019 ‘Students working in term-time: Commuter students and their working patterns’

The analysis by Newton is built upon the foundation of an examination system that might change its curriculum or change the style of questioning in an examination at various times. But in doing so, he also lays out a major reason why the whole system is inherently unfair to less advantaged groups. They may be in schools that are slow to adjust. Moving the goal posts favours those ready for such a change. It seems teachers and schools are being assessed as much as the students themselves. And they may, in turn have more or fewer resources to back them.

Note*: Paul Newton is no stranger to assessments. Currently the ‘Research Chair’ at Ofqual he was professor of educational assessment at the Institute of Education, University of London. His previous positions since 1994 have been Director Cambridge Assessment Network division, Cambridge Assessment; Ofqual Head of Assessment Research; Senior Research Officer, National Foundation for Educational Research; Research Officer, Pre-school Learning Alliance; Research Associate, King's College London; Research Officer AQA (then Associated Examining Board).

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.

Comments

Popular posts from this blog

Ofqual holding back information

Ofqual has responded to an FOI request from TEFS this week. They held a staggering twenty-nine board meetings since March. Despite promising the Parliamentary Education Committee over a month ago they would publish the minutes “shortly” after their meeting on 16th September, they are still not able to do so. They cite “exemption for information that is intended to be published in the future” for minutes that are in the “process of being approved for publication” . More concerning is they are also citing exemption under the “Public Interest Test”. This means they might not be published, and Ofqual will open themselves up to legal challenges. If both the Department for Education and Ofqual are prevented from being more open, then public interest will lie shattered on the floor and lessons will not be learned.  Ofqual finally responded to the TEFS Freedom of Information (FOI) request to publish the minutes of its board meetings on Tuesday. It should have been replied to by 17th Septembe

COVID-19, SAGE and the universities ‘document dump’

The recent release of several documents by SAGE all at once was described by one observer as a “dump of docs”. They relate to returning to education this autumn and are somewhat confusing as they illustrate the complexities of the challenges still to be tackled. But there is much not fully addressed. Outbreaks of COVID-19 at universities spilling into local communities might also trigger city-wide lock-downs and a bad reaction from the locals. The mass migration of students to their hometowns will spread the chaos wider afield as there seems to be little evidence of planning for this inevitability. Less advantaged students in poor accommodation or crowded homes will be at greater risk along with their vulnerable peers coping with health conditions. While students may be asked to ‘segment’ or form ‘bubbles’ staff might not have the same protection. Asking vulnerable lecturers and other staff with ongoing health conditions to move from classroom to classroom, contacting differen

Funding lifeline for disadvantaged students in schools under the spotlight

The image depicts the cover of a recent report by the Northern Ireland Audit Office that looks critically at the impact of ongoing additional school funding for disadvantaged students. Its hard-hitting conclusions must not be ignored. They show 15 years of failure and little impact despite nearly a billion spent across schools from 2005 to 2020. Similar schemes operate across the rest of the UK and the report raises serious questions about where the money is going. There is no doubt that disadvantages at home impact upon how students get on at school. But the danger is that some opponents will seize upon the findings to argue that the money should be withdrawn since it appears to do no good. Wiser heads will ask about where the money is going before reaching such a perverse conclusion. This is a time of considerable danger for those with few advantages. A wider social intervention will be needed to address the problems, and it is unreasonable to expect schools to impact things beyond