Teaching rankings in Europe and the experiences of students fuels a call for change.

The overarching feeling this week was that things are changing. The Times Higher Education survey and teaching rankings will come in for some considerable criticism. However, they have set out a major challenge for universities if they are to truly serve our society and redress the social imbalances. The important input from students in the survey is highly valuable in fuelling the debate. The earlier Stand Alone conference illustrated the pressing need for this through the direct experiences of students. Their voice should be more widely heard and appreciated.

This week saw two events at the University of Glasgow that will have a direct effect upon students and the provision of Higher Education.  The link between the events is more than just a coincidence or geographical. Very important factors that determine the success of many students in universities were under examination. The Times Higher Education Summit announced the ‘Teaching Rankings in Europe’ that are bound to fuel considerable debate and hopefully action. Preceding this was a conference led by the charity Stand Alone that supports estranged adults, including students. The testimony from students was the highlight of this. The result is two blog items for the price of one.

The Times Higher Education Teaching Excellence Summit.

This conference took place over three days in the impressively grand surroundings of Glasgow University. The setting itself reinforced the idea that education at this level is well established and here to stay for a long time. Delegates and speakers could feel confident in debating how education is delivered at various institutions. However, it is worth noting that the conference was intended for “Vice-chancellors, rectors, presidents, deputy presidents, vice-presidents and distinguished scholars representing higher education and governments along with industries and businesses with close links to both sectors” [1]

Controversy about the role of research in relation to teaching seems to divide opinion in debate. Observations that active researchers do not make the best teachers, whereas those that specialise in teaching make a better offering for students, are commonplace. Student opinion was left to that of Lauren McDougall, the current President of the Students’ Representative Council at Glasgow University. She bemoaned that lecturers are trained mostly in research and not teaching. In one sense she is right. To secure a lectureship it is necessary to concentrate on getting a PhD and a research reputation beyond that. However, criticism of this observation might be levelled by lecturers that all have to complete a Post Graduate Certificate of Higher Education over two years.  This is an excellent innovation championed by the Higher education Academy for many years. Its effect has been to challenge all lecturers to consider a variety of approaches to the education of their students.  Criticism of the lecture format, sometimes seen as didactic by its very nature, was made through calls for more interactive learning. This fuelled a robust debate that many of those that lecture/teach in universities today would find utterly bemusing. After all, the reality for most is that a variety of methods of teaching are used together, depending upon the subject taught and the numbers and needs of the students. There is no single fit solution. A good lecture, from a lecturer at the front edge of knowledge in the field, inspires confidence and can galvanise students into thinking more deeply about their subject. It can provide the focal point that works alongside other more interactive forms of teaching to give a well rounded experience.

Teaching rankings in Europe and what it tells us about UK Universities.

The conference itself provided the setting for the release of the eagerly awaited ‘Europe Teaching Rankings’ [2].  Releasing the results immediately after Croatia knocked England out of the world cup on Wednesday evening might have been inspired by the possibility of ironic extrapolation. A small country such as Croatia fields a team that defeats the more talented and diverse team from a much larger G8 nation. Rankings and reputation mean very little when actual performance trumps all. This is a new teaching ranking methodology that is driven by a comprehensive student survey intended to capture as many of the universities in Europe, with over 5000 students, as possible. Over 30,000 students were surveyed in 10 European countries. They were asked to score thirteen statements, on a scale of 1 to 10, such as “If a friend or family member were considering going to university, based on your experience, how likely or unlikely are you to recommend your university to them?” and “To what extent do you have the opportunity to interact with the faculty and teachers at your university as part of your learning experience?”.  The effectiveness and delivery of teaching is measured entirely by student responses in this respect. However, to a lesser extent the reputation of each university for teaching quality is considered separately.  All of the questions are listed in the appendix below and the methodology is described in full in Times Higher Education [3]

The overall ranking is composed of four main questions.

Engagement: Does the institution effectively engage with its students? This considers nine of the responses to the student questions and crucially counts as 40% of the final score. It is probably the most important and new aspect of the rankings to date.
Resources: Does the institution have the capacity to effectively deliver teaching? This relies on data from other sources ,such as HESA in the UK , and counts for 20%. It assesses staff-to-student ratio, papers-to-staff ratio and the general quality of its services.
Outcomes: Does the institution generate appropriate outputs for students? This again counts for 20% and falls back upon the earlier 2018 THE Academic Reputation Survey in relation to teaching and reported graduation rates. However, it also is fed by the remaining four student questions in relation to skills and the world of work
Environment: Is the teaching and learning environment inclusive. This also counts for 20% and relates to the gender balance in both the academic staff and the student cohort.

Resources and the divisions in the UK University rankings.

A very cursory glance at the rankings reveals quickly that the major Russell Group Universities rank highly. Most of the UK universities in the UK were involved and they are considered here as Russell Group, Pre-92 and Post-92 institutions.

Figure 1 (Left)  illustrates the relationship between the final combined scores for each institution and the individual scores in each category.  The first thing to emerge is that the Russell Group Institutions in blue safely inhabit the right hand side of the plots. Oxford and Cambridge are obvious outliers. The second thing to emerge is that the Pre-92 (red) and then the post-92 (green) universities fall in line behind them. There are some encouraging overlaps but the trend is stark. The link between resources and the final score and ranking is obvious as is the outcome that relies heavily upon reputation. However, there are interesting and encouraging signs that should make the better resourced universities sit up.  The important observation is that engagement and environment is little affected by the overall score and ranking across the board.

The influence of resources, such better staff to student ratios, might be expected to have a large effect on teaching when bearing in mind the wide range in scores observed.  

Figure 2 (below) looks at the relationship between resource and the influence on engagement, outcomes and environment.  Outcome is heavily weighted in favour of the better resourced universities. But surprisingly, there is very little relationship to the measure of engagement that is probably the most important aspect of teaching from a student’s perspective. 

What does this tell us?  

It seems that a large number of universities achieve solid engagement with much less resource. This means that it is reasonable to conclude that these institutions are more efficient at teaching. Their staff work harder at teaching. The better resourced institutions divert significant staff and time to research that feeds their reputations and position in this ranking. If the UK TEF scores are overlaid on this data then the position becomes very confused and this is not presented here.

Are all equal in funding?

All of the universities across GB are in receipt of funds roughly equivalent  to £9,250 per home student per year as fees or grants. Those with many international students have much more per student to deploy. Those eagle eyed enough will have spotted two institutions labelled by black symbols in Figures 1 and 2.  These are the Northern Ireland Universities (Queen’s University – a black triangle and Ulster University – a black square). They are notable as they operate in a very restricted financial environment. Current fees for Northern Ireland undergraduate  students are £4,160 per annum (a relatively small number of GB students pay £9,250 fees). The Northern Ireland Government makes up part of the deficit but this restricts income to around £6,000 per student per year. There is a relatively low number of international students to help with costs. The fact that both institutions are performing well alongside their peers might be looked at more closely. It appears that they are working much harder and much more efficiently than those in the rest of the UK. However, this must be offset by acknowledging the undoubted pressure on staff to perform and outperform their peers elsewhere.

Measuring the Environment.

The negative trend with respect to environment vs resources reflects a gender imbalance at some universities and this mirrors a similar trend across the elite European Universities in the survey [4].  Limiting the environment measure in this way is probably due to accessing useful data. Gender data is easy to acquire and is an obvious thing to assess. But environment means so much more to students from different backgrounds.  Those with disabilities look closely at how inclusive the teaching is and the support that they are likely to get. Students from low income backgrounds ,or with no family support, consider many environmental factors. A general trend is for such students to opt for universities nearer to home and more often chose the Pos-92 institutions. This provides some security and enables established part-time jobs to remain in place. The outcome for them is often lower paid graduate employment and greater student loan debt. The Russell Group Universities have been criticised many times for not having a diverse student intake and this fuels the persistent  low social mobility in the UK.

The call here is to seek to add other environment measures that show the extent of social mix, inclusivity, and part-time employment linked to attainment. This will require universities to gather more and better data on their students and their circumstances. There is likely to be resistance to this, but it would result in the higher ranked institutions becoming more inclusive.

Stand Alone and its pledge to support estranged students.

Perhaps a more important event for students took place in Glasgow on Monday. The Charity Stand Alone held a one day event looking at how to improve support for estranged students at university. Its stated mission is to “support and raise awareness about adults that are estranged from their family or children”.  This inevitably includes many students that run into difficulties with their families [5].

To secure better support they are promoting a ‘Pledge’ that all universities are being asked to sign up to. Many are now doing so and more are considering it.

This means that they are: “publically committing your institution to supporting students who are studying without the support or approval of a family network”. In reality they must: “Over a two-year period, your institution would work towards creating the right environment and conditions for estranged students to complete their course by improving support mechanisms in our four key areas”. 

These are: Finance, accommodation, mental health and outreach.

The meeting itself was mainly attended by university practitioners tasked with supporting students. This provided genuine insight into the experiences of staff in the frontline who were all encouraged to join the discussion. The moves by Glasgow University in signing the ‘Pledge’, and in actively encouraging students to ‘self-report’ their circumstances, are to be commended. The Stand Alone CEO, Becca Bland exuded a quiet determination to get universities to expand their support and was clearly confident. One practical action discussed was to encourage everyone to treat such students in the same way as those that may be ‘care experienced’ during their earlier years. These students are more readily identified in official records and can expect more help. Others in materially similar difficulties are dropping well below that radar and must be actively sought out.

Notably, there was a gender balance amongst delegates that was weighted mainly in favour of women. This appears to be inevitable in our current environment and more men should be encouraged to take part. Also, the lessons coming from the meeting may not transmit easily to many of the academic staff that the students encounter every day. There was an overwhelming feeling that the issues discussed should be more widely disseminated in institutions. This might involve more awareness included in courses for lecturers such as the post graduate certificates of higher education that are now compulsory,. 

A breath of fresh air from students.

The Stand Alone meeting literally stood alone and head and shoulders above many other meetings that consider the fate of students in Higher Education. Unlike the larger meeting later in the week, there were presentations and evidence from students themselves affected by estrangement. Their testimony and determination was a  'breath of fresh air' that delegates at the THE Teaching Conference could have  benefited from taking in.  They represent the reality that both staff and students face every day. The role of students themselves is coming more to the fore and working closely with the NUS there will be an 'Estranged Students Solidarity Week 2018’ between the 26 - 30 November of this year [6]. This could open the door wider for many students that have not reported their problems.

We learned that the term ‘estrangement’ itself causes confusion as to what it really means. Indeed it can come about for many reasons. A student may find themselves without any financial support from their families. This is often not reported and they struggle on. There could have been a family break up, bereavement, disagreement about future careers or simply they are pushed out. Stand Alone  describes the many scenarios in its Blog [7]. They largely move amongst their peers unnoticed until someone asks questions such as about what they are planning for Christmas or the summer vacation. A common theme is described well by one student.

“I don't have a "safety net" if something goes wrong with studies/work and I can never really just take a break. I think that makes me more anxious about the future and generally more stress-prone.”

The idea that all families support their children is a comfortable notion for most students and staff but does not match the experience of many other students. The discussion with the students was illuminating in so many ways. One student described the testimony of others in a study. The experiences from some of these  students brought back some stark memories for me.  

“When I was moving in I didn’t actually have the money for the train ticket, so I just got on the train and prayed that no one asked me for my ticket”

“Perhaps the most difficult part of it all is the lack of a place to call ‘home’”

“Dissimilar to peers -‘we’re at two different heights’”

It is often not appreciated that complete estrangement may not be the problem in every case. There may be some contact with family but no financial support forthcoming. Emotional support, understanding and encouragement may also amount to zero. When asked about financial requirements in the absence of family support, it emerged that part-time jobs were an essential part of their survival. One was about to embark on a medicine degree at Glasgow but was fortunate that the level of support offered would negate the need to find employment. Guaranteed accommodation was a key factor.  Indeed this was necessary. Another graduate had to work throughout her degree but limited this to weekends only. The pressure on her was immense and she had to be resilient to do so well. Another about to start a degree course just accepted that part-time working would be needed.  The offer of someone to talk to and help with securing accommodation for 52 weeks of the year was essential for all. Financial pressures and poor accommodation looms for many students at what is a stressful time. 

Conclusions and hope for change on the horizon.

The overarching feeling this week was that things are changing. The Times Higher Education survey and teaching rankings will come in for some considerable criticism. This may be simply an aversion to ranking per se and problems with the methodology. However, they have set out a major challenge for universities if they are to truly serve our society and redress the social imbalances. The important input from students in the survey is highly valuable in fuelling the debate. The recent release of the Student Experience Survey 2018 [8] from Advance HE and HEPI provides a valuable insight into the lives of many students in the UK. However, its overall reporting approach falls short of giving data on individual institutions that could contribute to or provide a ranking. Merging this data with that of the THE survey in UK terms would be highly valuable. Also, the data that feeds the rankings could be improved to include other environmental factors related to widening participation, inclusivity and support for less advantaged students.  The Stand Alone conference illustrated the pressing need for this through the direct experiences of students.

Their voice should be more widely heard and appreciated.

Mike Larkin, is an Emeritus Professor of Microbial Biochemistry. He retired from Queen's University Belfast after 37 years teaching Microbiology, Biochemistry and Genetics.

References.

[1] Times Higher Education Teaching Excellence Summit 2018 Glasgow University http://www.theworldsummitseries.com/events/the-teaching-excellence-summit-2018/agenda-5d1f0ada43d742f78f9cf1faac07c7f0.aspx
[2] THE Europe Teaching Rankings 2018: July 11, 2018. https://www.timeshighereducation.com/world-university-rankings/europe-teaching-rankings-2018-results-announced
Full results at: https://www.timeshighereducation.com/rankings/europe-teaching/2018#!/page/0/length/25/sort_by/rank/sort_order/asc/cols/undefined
[3] THE Europe Teaching Rankings 2018: Methodology. https://www.timeshighereducation.com/world-university-rankings/europe-teaching-rankings-2018-methodology
[4] Prestigious European universities have ‘worst gender balance’. Times Higher Education July 12, 2018. https://www.timeshighereducation.com/fr/news/prestigious-european-universities-have-worst-gender-balance
[5] Stand Alone web site: https://www.standalone.org.uk/. Support for students: https://www.standalone.org.uk/students/
[6] 'Estranged Students Solidarity Week' 2018: 26 - 30 November 2018. http://www.thestandalonepledge.org.uk/students/su-campaign
[7] Stand Alone Blog ‘Hidden and overlooked: estranged young people in higher education’. htp://www.sfc.ac.uk/news/blogs/Blog-67714.aspx
[8] HEPI / Advance HE Student Academic Experience Survey 2018 http://www.hepi.ac.uk/wp-content/uploads/2018/06/STRICTLY-EMBARGOED-UNTIL-THURSDAY-7-JUNE-2018-Student-Academic-Experience-Survey-report-2018.pdf

Appendix. THE Student Survey questions

·         To what extent do you have the opportunity to interact with the faculty and teachers at your university as part of your learning experience?

To what extent does your university provide opportunities for collaborative learning?

·         To what extent does the teaching at your university support critical thinking?

 To what extent does the teaching at your university support reflection upon, or making connections among, things you have learned?

·         To what extent does the teaching at your university support applying your learning to the real world?

 To what extent have the classes you have taken in your university so far challenged you?

·         If a friend or family member were considering going to university, based on your experience, how likely or unlikely are you to recommend your university to them?

 Do you think your university is effective in helping you to secure valuable internships that prepare you for your chosen career?

·         To what extent are you satisfied with the access to learning materials (such as library, texts, labs and online material) at your university?

To what extent are you satisfied with the quality of the learning environment (such as lecture halls, study areas, and labs) at your university?

·         To what extent do you think that the teaching at your university is supporting you to learn skills that will be useful in the world of work after university (for example writing, numeric, presentation or other skills)?

 To what extent do you have the opportunity to learn about and make connections to the world of work after university (for example through work placements, careers advice)?