Skip to main content

FOUR: Are the teachers or the students being assessed?

It seems to be generally accepted that there are major differences between schools and colleges in the success of their students in examinations. One cause could be some schools and colleges have students with better support and advantages at home. This makes such differentials inevitable. Bearing in mind better off families are willing to shell out considerable sums of money to use independent schools, it is likely that teaching and resources combine to lead to success. All things being equal otherwise, it looks like the school and teaching itself is under the microscope when examinations come along.  Hence the obsession with 'league tables'. Thus, the teacher’s ability is being assessed. The actual ability of the students seems relegated to a ghost-like’ presence hovering in the wings. 

To accompany the conclusion of no increased equality bias with examinations, Ofqual added a report on its underlying mission on standards (Monday 23rd November 2020 ‘Maintaining Standards’). This is an interesting treatise by Paul Newton, from Ofqual’s Strategy, Risk, and Research Directorate (see Note* below), who considered the logic underpinning maintenance of ‘standards’ during “normal times”, and by way of a warning about what happens “when qualifications are reformed”. There are disturbing aspects of the article that hint at the main problem of inequality in the system. 

The Sawtooth effect. 

Firstly, Newton examines in hypothetical terms the so called ‘sawtooth’ effect before and after there are reforms to examination processes. It simply shows that the performance of students in exams tends to rise over time as teachers become more familiar with the examinations and its tendencies. However, this drops immediately after the examinations are reformed. Then, performance rises again over time as teachers adapt to the new regime. This leads to shifting of grade boundaries relative to the actual performance in the old and new examinations, with boundaries drifting up over time. 

Secondly, the treatise is an unintended admission that teaching quality has a bigger influence on the outcome of examinations over time. This lies at the root of inequality whereby schools with smaller classes, more and better resources and students with family support react faster to the changes. Those at a disadvantage are slow to catch up and it explains why many parents are so wedded to using expensive, well-resourced independent schools. With exam topics released to schools in advance in England in 2021, it is not difficult to imagine those schools ramping up their provision and tutoring fast. That is what the parents pay for. There could be a significant adverse impact on attainment gap next year. 

Instead of this, on fairness Newton poses the question “If the performance of the first cohort post-reform is lower than the performance of the last cohort pre-reform, then should the new cohort of students end up with lower grades”. That some better resourced schools react faster to change is not considered. The answer is that it would not seem fair if standards are to be maintained year on year. Despite differences in ‘Attainment-Referencing’ the exams boards have resorted to making adjustments on the ‘Comparable Outcomes’ principle. However, Newton concedes “There are many outstanding challenges to address – both practical and theoretical – and it is possible that some of these challenges may never be solved.” 

Authentic or inauthentic gains. 

It could be argued that the students entering newly reformed examinations as the first cohort are largely all in the same boat. They might all be equally ill prepared. However, over time their teachers learn to adapt through what Newton calls “realignment”. This is accompanied by “adeptness” or “gains in cohort performance, over the first few years of reformed assessment arrangements, which are attributable solely to teachers becoming better at preparing learners for the new assessment structure/formats.” 

Newton makes a distinction between “authentic” gains in performance based upon content and “inauthentic” gains related to the “process by which it is assessed”. As a university teacher, I preferred students who were better informed and prepared for a performance driven assessment system. For students, this is also better rather than being adept at gaming an examination system that is radically different in a university. 

The problem I often found was that performance is affected by lack of resources and time to study, particularly those working part-time and/or commuting (TEFS 23rd August 2019 ‘Students working in term-time: Commuter students and their working patterns’

The analysis by Newton is built upon the foundation of an examination system that might change its curriculum or change the style of questioning in an examination at various times. But in doing so, he also lays out a major reason why the whole system is inherently unfair to less advantaged groups. They may be in schools that are slow to adjust. Moving the goal posts favours those ready for such a change. It seems teachers and schools are being assessed as much as the students themselves. And they may, in turn have more or fewer resources to back them.

Note*: Paul Newton is no stranger to assessments. Currently the ‘Research Chair’ at Ofqual he was professor of educational assessment at the Institute of Education, University of London. His previous positions since 1994 have been Director Cambridge Assessment Network division, Cambridge Assessment; Ofqual Head of Assessment Research; Senior Research Officer, National Foundation for Educational Research; Research Officer, Pre-school Learning Alliance; Research Associate, King's College London; Research Officer AQA (then Associated Examining Board).

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.

Comments

Popular posts from this blog

Bristol University student death: Inquest raises many concerns

The inquest into the tragic death of Bristol University Student, Ben Murray, took place this week; almost 12 months since he took his own life.* The coroner recorded a verdict of suicide earlier today but warned the University that it should make detailed inquiries after each death (BBC News ‘University of Bristol told to learn lessons after Ben Murray's suicide’ ). The anniversary of his death is this Sunday the 5th of May. Spring comes as a time of hope for most people but for others it can be a time of considerable anxiety and stress. This is especially the case for students approaching the examination period. As a close colleague of mine often pointed out, “they are all someone’s child”. Our hearts go out to the family of Ben Murray and friends as the inquest goes over again the events of a year ago. The pain is further exacerbated by media reports that he had little or no support in what was his first year at university. The BBC reported that ‘Bristol University studen

Ofqual holding back information

Ofqual has responded to an FOI request from TEFS this week. They held a staggering twenty-nine board meetings since March. Despite promising the Parliamentary Education Committee over a month ago they would publish the minutes “shortly” after their meeting on 16th September, they are still not able to do so. They cite “exemption for information that is intended to be published in the future” for minutes that are in the “process of being approved for publication” . More concerning is they are also citing exemption under the “Public Interest Test”. This means they might not be published, and Ofqual will open themselves up to legal challenges. If both the Department for Education and Ofqual are prevented from being more open, then public interest will lie shattered on the floor and lessons will not be learned.  Ofqual finally responded to the TEFS Freedom of Information (FOI) request to publish the minutes of its board meetings on Tuesday. It should have been replied to by 17th Septembe

The perfect storm for Universities PART TWO: The COVID-19 ‘time bomb blind-spot’

Pdf LINK PART ONE set out the context of the mounting predicament universities are finding themselves in around the rise in student numbers coming down the line. PART TWO looks at the immediate burden of more students finding themselves in financial difficulty. Loss of income sources for many students will be compounded by families at home losing their incomes as the recession bites before Christmas. It will impact 'middle-class' families unused to the idea of poverty and add to the growing numbers of students seeking help. The government and universities may be stumbling into another storm by failing to see the extent of the problem because of a ‘blind spot’ in their understanding. TEFS has received reassurances directly from each of the UK Universities ministers, but they are putting too much faith in university administered hardship funds as their only fallback position. This brings many problems with it as the ministers reject a TEFS call for a UK wide task force on