Skip to main content

Examinations and ‘the ghost in the machine’

The title refers to the false notion that it is possible to fairly and reliably assess the ability of students to decide on who gets a university place. The COVID crisis has revealed serious shortcomings in how we might assess the ability and potential of students in an unequal education system that favours attainment achieved by those with better resources. Measuring the 'ghost in the machine’ that is someone’s real ability taken in context is not considered and this disadvantages those with fewer learning resources or support at home. 

Here are a series of posts relating to the final school and college examinations in the UK that determine transition to higher education. Whilst the governments in Scotland and Wales have cancelled their summer higher, advanced higher and A-level examinations, the governments in England and Northern Ireland will press on with examinations delayed by a few weeks next summer. The Northern Ireland government may have little choice since many students take A-level examinations set by the various boards in England. The decision therefore rests with the Government in England. At a hearing of the Education Committee this week, Schools Minister, Nick Gibb and acting Chief Regulator at Ofqual, Glenys Stacey pressed home their determination to proceed in the face of considerable criticism levelled about the impacts on disadvantaged students. The poor supply of computers needed for those learning from home was not defended well. 


Looks at the defence by Ofqual of its procedures this summer in the context of the current situation in the UK. It appears that Ofqual are still steering policy and are exerting their influence. This provides context for the decisions made by the government in England to carry on with examinations next summer. The signs are not good, and if examinations are cancelled again, it seems the same mistakes could be made.


Considers why Ofqual thinks its methods this summer were fair and did not show bias against disadvantaged groups. However, a closer look shows this may remain hidden in the way the data is presented and the lack of reliability in the examination marking process itself. Ofqual should be more open in how it looks at the data.


Looks at the inherent problem of the examination marking processes being unreliable. The COVID crisis and move to Centre Assessment Grades (CAGs) uncovered uncomfortable facts about how it is done in 'normal' times. The goal of maintaining standards as the main aim, at the expense of equality and fairness, has led to a waning of confidence in the system. 


Is a question we must ask. The answer appears to be that schools or colleges and their teachers are really being assessed in examinations taken by their students. This idea emerged with the use of CAGs that determined the grades of individual students based upon the past performance of the school or college. Observations by Ofqual of a ‘sawtooth’ effect in performance before and after reforms of examinations illustrates the extent to which the effectiveness of teachers and schools influences results. 


Examines the ‘behaviourist’ philosophy that appears to underpin the blind reliance on single examinations. It presupposes that there is an inherent ability in every individual student that can be readily measured. This manifests itself in looking narrowly at attainment that is tested in a single ‘one-off’ examination. But the conclusion that marking examinations is unreliable was bound to emerge. It ignores other data available on a student, the context of the learning environment and the many less tangible attributes that make us who we are. It seems the ‘ghost in the machine’ is ignored and left sitting on the bench waiting to be called upon to show its ability.

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.

Comments

Popular posts from this blog

Ofqual holding back information

Ofqual has responded to an FOI request from TEFS this week. They held a staggering twenty-nine board meetings since March. Despite promising the Parliamentary Education Committee over a month ago they would publish the minutes “shortly” after their meeting on 16th September, they are still not able to do so. They cite “exemption for information that is intended to be published in the future” for minutes that are in the “process of being approved for publication” . More concerning is they are also citing exemption under the “Public Interest Test”. This means they might not be published, and Ofqual will open themselves up to legal challenges. If both the Department for Education and Ofqual are prevented from being more open, then public interest will lie shattered on the floor and lessons will not be learned.  Ofqual finally responded to the TEFS Freedom of Information (FOI) request to publish the minutes of its board meetings on Tuesday. It should have been replied to by 17th Sept...

Higher Education and the ‘intelligent plumbers’ theory

A common tactic when found out is to divert attention elsewhere. The release of student data from 2018/19 by the Department for Education (DfE) yesterday, ‘Widening participation in higher education: 2020’ and ‘Statistics: further education and skills’ tells the same sorry tale of a wide gap in access to universities between the most and least advantaged students. To divert attention from these stark facts in advance, the government used a diversionary tactic by attacking the effectiveness of universities and thus pointing the blame for poor social mobility someplace else. Advocating improvements in further education, something cut back by the same regime for years, hides the real intention. It seems that class divisions will be further exacerbated and any concession to universities fuelling improved social mobility has been abandoned. But the flawed theory is that at least the elite rulers will get access to intelligent plumbers . Three years ago, I heard a leading ‘You...

Students working in term-time: Commuter students and their working patterns

This article and analysis shows that commuter students are more likely to be employed in term time and also more likely to work longer hours. Two recent studies of commuter students ( one a quantitative and the other a qualitative analysis ) attending six universities in the London area revealed that commuter students were at a disadvantage in terms of outcome when compared to their peers. There is an urgent need for institutions to consider the actual time that their students have to study as the main measure. This is a way to integrate the time pressures of other activities such as commuting and employment that all add up to less time for studying. The general conclusion of the two studies was that “travel time remained a significant predictor of student progression or continuation for UK-domiciled full time undergraduates at three of the six London institutions”. This is perhaps not surprising for someone who spends much of the day travelling and the recommendation is that ...