Skip to main content

Examinations and ‘the ghost in the machine’

The title refers to the false notion that it is possible to fairly and reliably assess the ability of students to decide on who gets a university place. The COVID crisis has revealed serious shortcomings in how we might assess the ability and potential of students in an unequal education system that favours attainment achieved by those with better resources. Measuring the 'ghost in the machine’ that is someone’s real ability taken in context is not considered and this disadvantages those with fewer learning resources or support at home. 

Here are a series of posts relating to the final school and college examinations in the UK that determine transition to higher education. Whilst the governments in Scotland and Wales have cancelled their summer higher, advanced higher and A-level examinations, the governments in England and Northern Ireland will press on with examinations delayed by a few weeks next summer. The Northern Ireland government may have little choice since many students take A-level examinations set by the various boards in England. The decision therefore rests with the Government in England. At a hearing of the Education Committee this week, Schools Minister, Nick Gibb and acting Chief Regulator at Ofqual, Glenys Stacey pressed home their determination to proceed in the face of considerable criticism levelled about the impacts on disadvantaged students. The poor supply of computers needed for those learning from home was not defended well. 

Looks at the defence by Ofqual of its procedures this summer in the context of the current situation in the UK. It appears that Ofqual are still steering policy and are exerting their influence. This provides context for the decisions made by the government in England to carry on with examinations next summer. The signs are not good, and if examinations are cancelled again, it seems the same mistakes could be made.

Considers why Ofqual thinks its methods this summer were fair and did not show bias against disadvantaged groups. However, a closer look shows this may remain hidden in the way the data is presented and the lack of reliability in the examination marking process itself. Ofqual should be more open in how it looks at the data.

Looks at the inherent problem of the examination marking processes being unreliable. The COVID crisis and move to Centre Assessment Grades (CAGs) uncovered uncomfortable facts about how it is done in 'normal' times. The goal of maintaining standards as the main aim, at the expense of equality and fairness, has led to a waning of confidence in the system. 

Is a question we must ask. The answer appears to be that schools or colleges and their teachers are really being assessed in examinations taken by their students. This idea emerged with the use of CAGs that determined the grades of individual students based upon the past performance of the school or college. Observations by Ofqual of a ‘sawtooth’ effect in performance before and after reforms of examinations illustrates the extent to which the effectiveness of teachers and schools influences results. 

Examines the ‘behaviourist’ philosophy that appears to underpin the blind reliance on single examinations. It presupposes that there is an inherent ability in every individual student that can be readily measured. This manifests itself in looking narrowly at attainment that is tested in a single ‘one-off’ examination. But the conclusion that marking examinations is unreliable was bound to emerge. It ignores other data available on a student, the context of the learning environment and the many less tangible attributes that make us who we are. It seems the ‘ghost in the machine’ is ignored and left sitting on the bench waiting to be called upon to show its ability.

Mike Larkin, retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics. He has served on the Senate and Finance and planning committee of a Russell Group University.


Popular posts from this blog

Ofqual holding back information

Ofqual has responded to an FOI request from TEFS this week. They held a staggering twenty-nine board meetings since March. Despite promising the Parliamentary Education Committee over a month ago they would publish the minutes “shortly” after their meeting on 16th September, they are still not able to do so. They cite “exemption for information that is intended to be published in the future” for minutes that are in the “process of being approved for publication” . More concerning is they are also citing exemption under the “Public Interest Test”. This means they might not be published, and Ofqual will open themselves up to legal challenges. If both the Department for Education and Ofqual are prevented from being more open, then public interest will lie shattered on the floor and lessons will not be learned.  Ofqual finally responded to the TEFS Freedom of Information (FOI) request to publish the minutes of its board meetings on Tuesday. It should have been replied to by 17th Septembe

COVID-19, SAGE and the universities ‘document dump’

The recent release of several documents by SAGE all at once was described by one observer as a “dump of docs”. They relate to returning to education this autumn and are somewhat confusing as they illustrate the complexities of the challenges still to be tackled. But there is much not fully addressed. Outbreaks of COVID-19 at universities spilling into local communities might also trigger city-wide lock-downs and a bad reaction from the locals. The mass migration of students to their hometowns will spread the chaos wider afield as there seems to be little evidence of planning for this inevitability. Less advantaged students in poor accommodation or crowded homes will be at greater risk along with their vulnerable peers coping with health conditions. While students may be asked to ‘segment’ or form ‘bubbles’ staff might not have the same protection. Asking vulnerable lecturers and other staff with ongoing health conditions to move from classroom to classroom, contacting differen